Changes in README.utf8data

66715f00 - unicode: change the reference of database file

Thu, 12 Sep 2024 03:19:32 +0000

unicode: change the reference of database fileCommit 2b3d04787012 ("unicode: Add utf8-data module") changedthe database file from 'utf8data.h' to 'utf8data.c' to buildseparate module, but it seems forgot to update README.utf8data, which may causes confusion. Update the README.utf8data andthe default 'UTF8_NAME' in 'mkutf8data.c'.Signed-off-by: Gan Jie Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20240912031932.1161-1-ganjie182@gmail.comSigned-off-by: Gabriel Krisman Bertazi List of files: /linux-6.15/fs/unicode/README.utf8data

7fb64133 - unicode: update to Unicode 12.1.0 final

Sun, 12 May 2019 17:26:08 +0000

unicode: update to Unicode 12.1.0 finalSigned-off-by: Theodore Ts'o Cc: Gabriel Krisman Bertazi List of files: /linux-6.15/fs/unicode/README.utf8data

28ba53c0 - unicode: refactor the rule for regenerating utf8data.h

Sun, 28 Apr 2019 17:45:36 +0000

unicode: refactor the rule for regenerating utf8data.hscripts/mkutf8data is used only when regenerating utf8data.h,which never happens in the normal kernel build. However, it isirrespectively built if CONFIG_UNICODE is enabled.Moreover, there is no good reason for it to reside in the scripts/directory since it is only used in fs/unicode/.Hence, move it from scripts/ to fs/unicode/.In some cases, we bypass build artifacts in the normal build. Theconventional way to do so is to surround the code with ifdef REGENERATE_*.For example, - 7373f4f83c71 ("kbuild: add implicit rules for parser generation") - 6aaf49b495b4 ("crypto: arm,arm64 - Fix random regeneration of S_shipped")I rewrote the rule in a more kbuild'ish style.In the normal build, utf8data.h is just shipped from the check-in file.$ make [ snip ] SHIPPED fs/unicode/utf8data.h CC fs/unicode/utf8-norm.o CC fs/unicode/utf8-core.o CC fs/unicode/utf8-selftest.o AR fs/unicode/built-in.aIf you want to generate utf8data.h based on UCD, put *.txt files intofs/unicode/, then pass REGENERATE_UTF8DATA=1 from the command line.The mkutf8data tool will be automatically compiled to generate theutf8data.h from the *.txt files.$ make REGENERATE_UTF8DATA=1 [ snip ] HOSTCC fs/unicode/mkutf8data GEN fs/unicode/utf8data.h CC fs/unicode/utf8-norm.o CC fs/unicode/utf8-core.o CC fs/unicode/utf8-selftest.o AR fs/unicode/built-in.aI renamed the check-in utf8data.h to utf8data.h_shipped so that thiswill work for the out-of-tree build.You can update it based on the latest UCD like this:$ make REGENERATE_UTF8DATA=1 fs/unicode/$ cp fs/unicode/utf8data.h fs/unicode/utf8data.h_shippedAlso, I added entries to .gitignore and dontdiff.Signed-off-by: Masahiro Yamada Signed-off-by: Theodore Ts'o List of files: /linux-6.15/fs/unicode/README.utf8data

1215d239 - unicode: update unicode database unicode version 12.1.0

Thu, 25 Apr 2019 17:59:17 +0000

unicode: update unicode database unicode version 12.1.0Regenerate utf8data.h based on the latest UCD files and run testsagainst the latest version.Signed-off-by: Gabriel Krisman Bertazi Signed-off-by: Theodore Ts'o List of files: /linux-6.15/fs/unicode/README.utf8data

a8384c68 - unicode: reduce the size of utf8data[]

Thu, 25 Apr 2019 17:49:18 +0000

unicode: reduce the size of utf8data[]Remove the Hangul decompositions from the utf8data trie, and doalgorithmic decomposition to calculate them on the fly. To store thedecomposition the caller of utf8lookup()/utf8nlookup() must provide a12-byte buffer, which is used to synthesize a leaf with thedecomposition. This significantly reduces the size of the utf8data[]array.Changes made by Gabriel: Rebase to mainline Fix checkpatch errors Extract robustness fixes and merge back to original mkutf8data.c patch Regenerate utf8data.hSigned-off-by: Olaf Weber Signed-off-by: Gabriel Krisman Bertazi Signed-off-by: Theodore Ts'o List of files: /linux-6.15/fs/unicode/README.utf8data

955405d1 - unicode: introduce UTF-8 character database

Thu, 25 Apr 2019 17:38:44 +0000

unicode: introduce UTF-8 character databaseThe decomposition and casefolding of UTF-8 characters are described in aprefix tree in utf8data.h, which is a generate from the UnicodeCharacter Database (UCD), published by the Unicode Consortium, andshould not be edited by hand. The structures in utf8data.h are meant tobe used for lookup operations by the unicode subsystem, when decoding autf-8 string.mkutf8data.c is the source for a program that generates utf8data.h. Itwas written by Olaf Weber from SGI and originally proposed to be mergedinto Linux in 2014. The original proposal performed the compatibilitydecomposition, NFKD, but the current version was modified by me to docanonical decomposition, NFD, as suggested by the community. Thechanges from the original submission are: * Rebase to mainline. * Fix out-of-tree-build. * Update makefile to build 11.0.0 ucd files. * drop references to xfs. * Convert NFKD to NFD. * Merge back robustness fixes from original patch. Requested by Dave Chinner.The original submission is archived at:The utf8data.h file can be regenerated using the instructions infs/unicode/README.utf8data.- Notes on the update from 8.0.0 to 11.0:The structure of the ucd files and special cases have not experiencedany changes between versions 8.0.0 and 11.0.0. 8.0.0 saw the additionof Cherokee LC characters, which is an interesting case forcase-folding. The update is accompanied by new tests on the test_ucdmodule to catch specific cases. No changes to mkutf8data script wererequired for the updates.Signed-off-by: Gabriel Krisman Bertazi Signed-off-by: Theodore Ts'o List of files: /linux-6.15/fs/unicode/README.utf8data