History log of /sqlite-3.40.0/ext/fts3/unicode/mkunicode.tcl (Results 1 – 24 of 24)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: release, version-3.50.2, version-3.50.1, major-release, version-3.50.0, version-3.49.2, patch-release, version-3.44.4, version-3.49.1, version-3.49.0, major-relase, relase, version-3.48.0, version-3.47.2, version-3.47.1, version-3.47.0, version-3.46.1, version-3.46.0, version-3.45.3, version-3.44.3, version-3.45.2, version-3.45.1, vesion-3.45.1, version-3.45.0, version-3.44.2, version-3.44.1, version-3.44.0, version-3.43.2, version-3.43.1, version-3.43.0, version-3.42.0, version-3.41.2, version-3.41.1, version-3.41.0, version-3.40.1, version-3.40.0, version-3.39.4, version-3.39.3, version-3.39.2, version-3.39.1, version-3.39.0, version-3.38.5, version-3.38.4, relese, version-3.38.3, version-3.38.2, version-3.38.1, version-3.38.0, version-3.37.2, version-3.37.1, version-3.37.0, version-3.36.0, version-3.35.5, version-3.35.4, version-3.35.3, same-as-3.35.3, version-3.35.2, version-3.35.1, version-3.35.0, patch, version-3.34.1, version-3.34.0
# ec896286 26-Nov-2020 dan <Dan Kennedy>

Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9].

FossilOrigin-Name: 326d579d777fdede6bc64f9525248767f4730de4e50260b0387e614a9d006416


Revision tags: version-3.33.0, version-3.32.3, version-3.32.2, version-3.32.1, version-3.32.0, version-3.31.1, version-3.31.0, version-3.30.1, version-3.30.0, version-3.29.0, version-3.28.0
# 065f3bf4 20-Mar-2019 mistachkin <[email protected]>

Fix various harmless compiler warnings seen with MSVC.

FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c


Revision tags: version-3.27.2, version-3.27.1, version-3.27.0
# 8fc4a11c 02-Jan-2019 drh <[email protected]>

Fix harmless compiler warnings in the unicode2 logic of FTS3 and FTS5.

FossilOrigin-Name: 703029ac6d24860230a8c30fcbf5e7e1da619e84f1cc9b9e65ebc74879a184d2


# b163b572 28-Dec-2018 dan <[email protected]>

Fix problems in fts5 found by ASAN.

FossilOrigin-Name: c564bf870106faef297594a51995619c80311d06bd5f8a0c7644f666f22ba576


# f8c2fea1 03-Dec-2018 drh <[email protected]>

Remove the unused sqlite3Fts5UnicodeNCat() function.

FossilOrigin-Name: 7149dacf1d440a19f62808b4591c3fa8da202b2ec742d5490a63f2ec005ff9e7


# e89feee5 03-Dec-2018 dan <[email protected]>

Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5
and FTS3/4.

FossilOrigin-Name: 06177f3f114b5d804b84c27ac843740282e2176fdf0f7a999feda0e1b624adec


Revision tags: version-3.26.0, version-3.25.3, version-3.25.2, version-3.25.1, version-3.25.0
# b80bb6ce 13-Jul-2018 dan <[email protected]>

Add the "categories" option to the unicode61 tokenizer in fts5.

FossilOrigin-Name: 80d2b9e635e3100f90cffdcffa5b5038da6fbbfccc9f5777c59a4ae760d4cb62


Revision tags: version-3.24.0, version-3.23.2, version-3.23.1, version-3.23.0, version-3.22.0, version-3.21.0, version-3.20.1, version-3.19.4, version-3.20.0, version-3.18.2, version-3.18.1, version-3.19.3, version-3.19.2, version-3.19.1, version-3.19.0, version-3.18.0
# 920c83f1 20-Mar-2017 dan <[email protected]>

Fix some problems in fts3 found by address-sanitizer.

FossilOrigin-Name: 16a8e84fa7f67a467f824bdd7f72cbd6a6e95dab8cc7aa1e0e751720b98f3e31


Revision tags: version-3.17.0, version-3.16.2, version-3.16.1, version-3.16.0, version-3.15.2, version-3.15.1, version-3.15.0, version-3.14.2, version-3.14.1, version-3.14.0, version-3.13.0, version-3.12.2, version-3.12.1, version-3.9.3, version-3.12.0, version-3.11.1, version-3.11.0
# 53ff9c29 12-Feb-2016 dan <[email protected]>

Fix a potential buffer overread provoked by invalid utf-8 in fts5.

FossilOrigin-Name: a049fbbde5da2e43d41aa8c2b41f9eb21507ac76


Revision tags: version-3.10.2, version-3.10.1, version-3.10.0, version-3.9.2, version-3.9.1, version-3.9.0, version-3.8.11.1, version-3.8.11
# 3f09beda 02-Jul-2015 dan <[email protected]>

Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.

FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39

Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.

FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436

show more ...


# 2e7d35e2 23-May-2015 dan <[email protected]>

Avoid making redundant copies of position-lists within the fts5 code.

FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0


# 21b7d2a9 22-May-2015 dan <[email protected]>

Improve test coverage of fts5_unicode2.c.

FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e


Revision tags: version-3.8.10.2, version-3.8.10.1, version-3.8.10, version-3.8.9, version-3.8.8.3
# 57fec54b 02-Feb-2015 dan <[email protected]>

Fix some problems with building fts5 and fts3 together using the amalgamation.

FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff


Revision tags: version-3.8.8.2, version-3.8.8.1, version-3.8.8
# 6024772b 01-Jan-2015 dan <[email protected]>

Add a version of the unicode61 tokenizer to fts5.

FossilOrigin-Name: d09f7800cf14f73ea86d037107ef80295b2c173a


Revision tags: version-3.8.7.4, version-3.8.7.3, version-3.8.7.2, version-3.8.7.1, version-3.8.6.1, version-3.8.7, version-3.8.6
# 858b638d 06-Aug-2014 drh <[email protected]>

A couple more harmless compiler warnings eliminated.

FossilOrigin-Name: bcf6d775f90f4d1ba018a1b965f2f710df130f01


# e8f2c9dc 06-Aug-2014 drh <[email protected]>

Fix two more harmless compiler warnings. Make sure the fts3_unicode2.c file
is in sync with mkunicode.tcl.

FossilOrigin-Name: a2a60307ea68a3230952a56cb65369ba0a208967


Revision tags: version-3.8.5, version-3.8.4.3, version-3.8.4.2, version-3.8.4.1, version-3.8.4, version-3.8.3.1, version-3.8.3, version-3.8.2, version-3.8.1, version-3.8.0.2, version-3.8.0.1, version-3.8.0
# f2c9229f 05-Jun-2013 dan <[email protected]>

Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit f

Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD.

FossilOrigin-Name: 6cfd9af5250029c0d275be027b4208c48954a8a1

show more ...


Revision tags: version-3.7.17, version-3.7.16.2, version-3.7.16.1, version-3.7.16, version-3.7.15.2, version-3.7.15.1, version-3.7.15, version-3.7.14.1, version-3.7.14, version-3.7.13
# 754d3adf 06-Jun-2012 dan <[email protected]>

Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".

FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0f

Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".

FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd

show more ...


# a9cfaba9 28-May-2012 drh <[email protected]>

Omit the fts3 unicode character class routines from the build if fts3/4
is disabled.

FossilOrigin-Name: c00bb5d4601efc15933f222349e96a043b610a19


# 7946c530 26-May-2012 dan <[email protected]>

If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.

FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054


# 501c74d3 26-May-2012 dan <[email protected]>

Change the format of the tables used by sqlite3FtsUnicodeTolower() to make them a little smaller.

FossilOrigin-Name: b89d3834f6690073fca0fc22c18afa1fb280ea7d


# 1c7016c9 25-May-2012 dan <[email protected]>

Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range.

FossilOrigin-Name: cf7b25d47687635a04f4347d45f135c686b9d758


# 80ed5a56 25-May-2012 dan <[email protected]>

Fix comments in generated file fts3_unicode2.c.

FossilOrigin-Name: 3dc567ef4702d9a63d78d11ff705cb7f7359f7a6


# 3d403c71 25-May-2012 dan <[email protected]>

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Lette

Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.

FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385

show more ...