History log of /sqlite-3.40.0/test/fts4unicode.test (Results 1 – 18 of 18)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: release, version-3.50.2, version-3.50.1, major-release, version-3.50.0, version-3.49.2, patch-release, version-3.44.4, version-3.49.1, version-3.49.0, major-relase, relase, version-3.48.0, version-3.47.2, version-3.47.1, version-3.47.0, version-3.46.1, version-3.46.0, version-3.45.3, version-3.44.3, version-3.45.2, version-3.45.1, vesion-3.45.1, version-3.45.0, version-3.44.2, version-3.44.1, version-3.44.0, version-3.43.2, version-3.43.1, version-3.43.0, version-3.42.0, version-3.41.2, version-3.41.1, version-3.41.0, version-3.40.1, version-3.40.0, version-3.39.4, version-3.39.3, version-3.39.2, version-3.39.1, version-3.39.0, version-3.38.5, version-3.38.4, relese, version-3.38.3, version-3.38.2, version-3.38.1, version-3.38.0, version-3.37.2, version-3.37.1, version-3.37.0, version-3.36.0, version-3.35.5, version-3.35.4, version-3.35.3, same-as-3.35.3, version-3.35.2, version-3.35.1, version-3.35.0, patch, version-3.34.1
# 31afee93 04-Jan-2021 dan <Dan Kennedy>

Add extra test for handling of embedded nul characters in the fts4 unicode61 tokenizer.

FossilOrigin-Name: c2c2c7e945f5d5700d91b8e779117e70e388ffc613912a434885ae27f5fe4e22


Revision tags: version-3.34.0, version-3.33.0, version-3.32.3, version-3.32.2, version-3.32.1, version-3.32.0, version-3.31.1, version-3.31.0, version-3.30.1, version-3.30.0, version-3.29.0, version-3.28.0, version-3.27.2, version-3.27.1, version-3.27.0, version-3.26.0, version-3.25.3, version-3.25.2, version-3.25.1, version-3.25.0, version-3.24.0, version-3.23.2, version-3.23.1, version-3.23.0, version-3.22.0, version-3.21.0, version-3.20.1, version-3.19.4, version-3.20.0, version-3.18.2, version-3.18.1, version-3.19.3
# d26d2c7d 30-May-2017 dan <[email protected]>

Omit a test of codepoint 0x202F (non-break narrow space) from the fts3 ICU
tests. Different versions of ICU apparently handle this obscure codepoint
slightly differently.

FossilOrigin-Name: 69ae6889

Omit a test of codepoint 0x202F (non-break narrow space) from the fts3 ICU
tests. Different versions of ICU apparently handle this obscure codepoint
slightly differently.

FossilOrigin-Name: 69ae688982d6cb9f859f5643c315a1dc5ba76ad35553ecea8329a75ee70a87b1

show more ...


Revision tags: version-3.19.2, version-3.19.1, version-3.19.0, version-3.18.0, version-3.17.0, version-3.16.2, version-3.16.1, version-3.16.0, version-3.15.2, version-3.15.1, version-3.15.0, version-3.14.2, version-3.14.1, version-3.14.0, version-3.13.0, version-3.12.2, version-3.12.1, version-3.9.3, version-3.12.0, version-3.11.1, version-3.11.0, version-3.10.2, version-3.10.1, version-3.10.0, version-3.9.2, version-3.9.1, version-3.9.0, version-3.8.11.1, version-3.8.11
# 07d694c7 15-Jun-2015 drh <[email protected]>

Adjust ICU tests to account for recent changes in the official
Unicode definition of whitespace.

FossilOrigin-Name: 0816525386ac51454b7b09a507e45b6a2cb8bf6e


Revision tags: version-3.8.10.2, version-3.8.10.1, version-3.8.10, version-3.8.9, version-3.8.8.3, version-3.8.8.2, version-3.8.8.1, version-3.8.8, version-3.8.7.4, version-3.8.7.3, version-3.8.7.2, version-3.8.7.1, version-3.8.6.1, version-3.8.7, version-3.8.6, version-3.8.5, version-3.8.4.3, version-3.8.4.2, version-3.8.4.1, version-3.8.4, version-3.8.3.1, version-3.8.3
# f6b1a8e1 19-Dec-2013 drh <[email protected]>

Make sure errors encountered while initializing extensions such as FTS4
get reported out from sqlite3_open(). This fixes a bug introduced by
check-in [9d347f547e7ba9]. Also remove lots of forgotten

Make sure errors encountered while initializing extensions such as FTS4
get reported out from sqlite3_open(). This fixes a bug introduced by
check-in [9d347f547e7ba9]. Also remove lots of forgotten "breakpoint"
commands left in test scripts over the years.

FossilOrigin-Name: ca3fdfd41961d8d3d1e39d20dc628e8a95dabb2f

show more ...


Revision tags: version-3.8.2, version-3.8.1
# 549bc3db 12-Oct-2013 mistachkin <[email protected]>

Fix Unicode character encoding issues on Windows.

FossilOrigin-Name: c9310c9a2bad11f1d033a57b33ea7aed43a8238d


# cbc53fec 11-Oct-2013 mistachkin <[email protected]>

Fix test numbering.

FossilOrigin-Name: cef39f6933dcfec4b4a087a05dbb4e7766003fb7


# 6284d021 18-Sep-2013 dan <[email protected]>

Test that the unicode61 tokenchars= and separators= options work with the fts3tokenize virtual table.

FossilOrigin-Name: ed24051462c09220ebfb82a347b4a2b5c820ef63


# f1d2670d 13-Sep-2013 dan <[email protected]>

Add tests for the fts4 unicode61 tokenchars and separators options.

FossilOrigin-Name: 9ce6f40dfb54b35cecba3cc9c1ec0d111f6e9f11


Revision tags: version-3.8.0.2
# 43398081 30-Aug-2013 dan <[email protected]>

Add a test for fts4 unicode61 option remove_diacritics=0.

FossilOrigin-Name: 6bf7ae6ff6b18712544ddeafb6848b3b27ff22d2


Revision tags: version-3.8.0.1, version-3.8.0
# f2c9229f 05-Jun-2013 dan <[email protected]>

Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit f

Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD.

FossilOrigin-Name: 6cfd9af5250029c0d275be027b4208c48954a8a1

show more ...


Revision tags: version-3.7.17, version-3.7.16.2, version-3.7.16.1, version-3.7.16
# 7c37e2f6 26-Jan-2013 drh <[email protected]>

Add a single test case to fts4unicode.test to verify that title-case
maps to lower case.

FossilOrigin-Name: 955a9459dabad231aa8d6282676975ab7fba244e


Revision tags: version-3.7.15.2, version-3.7.15.1, version-3.7.15, version-3.7.14.1, version-3.7.14
# 3aaa4cd9 19-Jun-2012 dan <[email protected]>

Add tests to check that the "unicode61" and "icu" tokenizers both identify white-space codepoints outside the ASCII range.

FossilOrigin-Name: bfb2d4730cbbe18fb940e72f4fde9122d550734e


Revision tags: version-3.7.13
# 25cdf46a 07-Jun-2012 dan <[email protected]>

Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.

FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc628

Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.

FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc62815c21e26a52

show more ...


# 754d3adf 06-Jun-2012 dan <[email protected]>

Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".

FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0f

Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".

FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd

show more ...


# 7946c530 26-May-2012 dan <[email protected]>

If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.

FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054


# 7a796731 26-May-2012 dan <[email protected]>

Add coverage tests for fts3_unicode.c.

FossilOrigin-Name: 07d3ea8a3cb179fab6c48934fc6751f53b507d36


# ab322bd2 26-May-2012 dan <[email protected]>

Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.

FossilOrigin-Name: 8f3e

Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.

FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060

show more ...


# 3d403c71 25-May-2012 dan <[email protected]>

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Lette

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.

FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385

show more ...