1*a9643ea8Slogwang 2*a9643ea8SlogwangXZ Embedded 3*a9643ea8Slogwang=========== 4*a9643ea8Slogwang 5*a9643ea8Slogwang XZ Embedded is a relatively small, limited implementation of the .xz 6*a9643ea8Slogwang file format. Currently only decoding is implemented. 7*a9643ea8Slogwang 8*a9643ea8Slogwang XZ Embedded was written for use in the Linux kernel, but the code can 9*a9643ea8Slogwang be easily used in other environments too, including regular userspace 10*a9643ea8Slogwang applications. See userspace/xzminidec.c for an example program. 11*a9643ea8Slogwang 12*a9643ea8Slogwang This README contains information that is useful only when the copy 13*a9643ea8Slogwang of XZ Embedded isn't part of the Linux kernel tree. You should also 14*a9643ea8Slogwang read linux/Documentation/xz.txt even if you aren't using XZ Embedded 15*a9643ea8Slogwang as part of Linux; information in that file is not repeated in this 16*a9643ea8Slogwang README. 17*a9643ea8Slogwang 18*a9643ea8SlogwangCompiling the Linux kernel module 19*a9643ea8Slogwang 20*a9643ea8Slogwang The xz_dec module depends on crc32 module, so make sure that you have 21*a9643ea8Slogwang it enabled (CONFIG_CRC32). 22*a9643ea8Slogwang 23*a9643ea8Slogwang Building the xz_dec and xz_dec_test modules without support for BCJ 24*a9643ea8Slogwang filters: 25*a9643ea8Slogwang 26*a9643ea8Slogwang cd linux/lib/xz 27*a9643ea8Slogwang make -C /path/to/kernel/source \ 28*a9643ea8Slogwang KCPPFLAGS=-I"$(pwd)/../../include" M="$(pwd)" \ 29*a9643ea8Slogwang CONFIG_XZ_DEC=m CONFIG_XZ_DEC_TEST=m 30*a9643ea8Slogwang 31*a9643ea8Slogwang Building the xz_dec and xz_dec_test modules with support for BCJ 32*a9643ea8Slogwang filters: 33*a9643ea8Slogwang 34*a9643ea8Slogwang cd linux/lib/xz 35*a9643ea8Slogwang make -C /path/to/kernel/source \ 36*a9643ea8Slogwang KCPPFLAGS=-I"$(pwd)/../../include" M="$(pwd)" \ 37*a9643ea8Slogwang CONFIG_XZ_DEC=m CONFIG_XZ_DEC_TEST=m CONFIG_XZ_DEC_BCJ=y \ 38*a9643ea8Slogwang CONFIG_XZ_DEC_X86=y CONFIG_XZ_DEC_POWERPC=y \ 39*a9643ea8Slogwang CONFIG_XZ_DEC_IA64=y CONFIG_XZ_DEC_ARM=y \ 40*a9643ea8Slogwang CONFIG_XZ_DEC_ARMTHUMB=y CONFIG_XZ_DEC_SPARC=y 41*a9643ea8Slogwang 42*a9643ea8Slogwang If you want only one or a few of the BCJ filters, omit the appropriate 43*a9643ea8Slogwang variables. CONFIG_XZ_DEC_BCJ=y is always required to build the support 44*a9643ea8Slogwang code shared between all BCJ filters. 45*a9643ea8Slogwang 46*a9643ea8Slogwang Most people don't need the xz_dec_test module. You can skip building 47*a9643ea8Slogwang it by omitting CONFIG_XZ_DEC_TEST=m from the make command line. 48*a9643ea8Slogwang 49*a9643ea8SlogwangCompiler requirements 50*a9643ea8Slogwang 51*a9643ea8Slogwang XZ Embedded should compile as either GNU-C89 (used in the Linux 52*a9643ea8Slogwang kernel) or with any C99 compiler. Getting the code to compile with 53*a9643ea8Slogwang non-GNU C89 compiler or a C++ compiler should be quite easy as 54*a9643ea8Slogwang long as there is a data type for unsigned 64-bit integer (or the 55*a9643ea8Slogwang code is modified not to support large files, which needs some more 56*a9643ea8Slogwang care than just using 32-bit integer instead of 64-bit). 57*a9643ea8Slogwang 58*a9643ea8Slogwang If you use GCC, try to use a recent version. For example, on x86-32, 59*a9643ea8Slogwang xz_dec_lzma2.c compiled with GCC 3.3.6 is 15-25 % slower than when 60*a9643ea8Slogwang compiled with GCC 4.3.3. 61*a9643ea8Slogwang 62*a9643ea8SlogwangEmbedding into userspace applications 63*a9643ea8Slogwang 64*a9643ea8Slogwang To embed the XZ decoder, copy the following files into a single 65*a9643ea8Slogwang directory in your source code tree: 66*a9643ea8Slogwang 67*a9643ea8Slogwang linux/include/linux/xz.h 68*a9643ea8Slogwang linux/lib/xz/xz_crc32.c 69*a9643ea8Slogwang linux/lib/xz/xz_dec_lzma2.c 70*a9643ea8Slogwang linux/lib/xz/xz_dec_stream.c 71*a9643ea8Slogwang linux/lib/xz/xz_lzma2.h 72*a9643ea8Slogwang linux/lib/xz/xz_private.h 73*a9643ea8Slogwang linux/lib/xz/xz_stream.h 74*a9643ea8Slogwang userspace/xz_config.h 75*a9643ea8Slogwang 76*a9643ea8Slogwang Alternatively, xz.h may be placed into a different directory but then 77*a9643ea8Slogwang that directory must be in the compiler include path when compiling 78*a9643ea8Slogwang the .c files. 79*a9643ea8Slogwang 80*a9643ea8Slogwang Your code should use only the functions declared in xz.h. The rest of 81*a9643ea8Slogwang the .h files are meant only for internal use in XZ Embedded. 82*a9643ea8Slogwang 83*a9643ea8Slogwang You may want to modify xz_config.h to be more suitable for your build 84*a9643ea8Slogwang environment. Probably you should at least skim through it even if the 85*a9643ea8Slogwang default file works as is. 86*a9643ea8Slogwang 87*a9643ea8SlogwangIntegrity check support 88*a9643ea8Slogwang 89*a9643ea8Slogwang XZ Embedded always supports the integrity check types None and 90*a9643ea8Slogwang CRC32. Support for CRC64 is optional. SHA-256 is currently not 91*a9643ea8Slogwang supported in XZ Embedded although the .xz format does support it. 92*a9643ea8Slogwang The xz tool from XZ Utils uses CRC64 by default, but CRC32 is usually 93*a9643ea8Slogwang enough in embedded systems to keep the code size smaller. 94*a9643ea8Slogwang 95*a9643ea8Slogwang If you want support for CRC64, you need to copy linux/lib/xz/xz_crc64.c 96*a9643ea8Slogwang into your application, and #define XZ_USE_CRC64 in xz_config.h or in 97*a9643ea8Slogwang compiler flags. 98*a9643ea8Slogwang 99*a9643ea8Slogwang When using the internal CRC32 or CRC64, their lookup tables need to be 100*a9643ea8Slogwang initialized with xz_crc32_init() and xz_crc64_init(), respectively. 101*a9643ea8Slogwang See xz.h for details. 102*a9643ea8Slogwang 103*a9643ea8Slogwang To use external CRC32 or CRC64 code instead of the code from 104*a9643ea8Slogwang xz_crc32.c or xz_crc64.c, the following #defines may be used 105*a9643ea8Slogwang in xz_config.h or in compiler flags: 106*a9643ea8Slogwang 107*a9643ea8Slogwang #define XZ_INTERNAL_CRC32 0 108*a9643ea8Slogwang #define XZ_INTERNAL_CRC64 0 109*a9643ea8Slogwang 110*a9643ea8Slogwang Then it is up to you to provide compatible xz_crc32() or xz_crc64() 111*a9643ea8Slogwang functions. 112*a9643ea8Slogwang 113*a9643ea8Slogwang If the .xz file being decompressed uses an integrity check type that 114*a9643ea8Slogwang isn't supported by XZ Embedded, it is treated as an error and the 115*a9643ea8Slogwang file cannot be decompressed. For multi-call mode, this can be modified 116*a9643ea8Slogwang by #defining XZ_DEC_ANY_CHECK. Then xz_dec_run() will return 117*a9643ea8Slogwang XZ_UNSUPPORTED_CHECK when unsupported check type is detected. After 118*a9643ea8Slogwang that decompression can be continued normally except that the 119*a9643ea8Slogwang integrity check won't be verified. In single-call mode there's 120*a9643ea8Slogwang no way to continue decoding, so XZ_DEC_ANY_CHECK is almost useless 121*a9643ea8Slogwang in single-call mode. 122*a9643ea8Slogwang 123*a9643ea8SlogwangBCJ filter support 124*a9643ea8Slogwang 125*a9643ea8Slogwang If you want support for one or more BCJ filters, you need to copy also 126*a9643ea8Slogwang linux/lib/xz/xz_dec_bcj.c into your application, and use appropriate 127*a9643ea8Slogwang #defines in xz_config.h or in compiler flags. You don't need these 128*a9643ea8Slogwang #defines in the code that just uses XZ Embedded via xz.h, but having 129*a9643ea8Slogwang them always #defined doesn't hurt either. 130*a9643ea8Slogwang 131*a9643ea8Slogwang #define Instruction set BCJ filter endianness 132*a9643ea8Slogwang XZ_DEC_X86 x86-32 or x86-64 Little endian only 133*a9643ea8Slogwang XZ_DEC_POWERPC PowerPC Big endian only 134*a9643ea8Slogwang XZ_DEC_IA64 Itanium (IA-64) Big or little endian 135*a9643ea8Slogwang XZ_DEC_ARM ARM Little endian only 136*a9643ea8Slogwang XZ_DEC_ARMTHUMB ARM-Thumb Little endian only 137*a9643ea8Slogwang XZ_DEC_SPARC SPARC Big or little endian 138*a9643ea8Slogwang 139*a9643ea8Slogwang While some architectures are (partially) bi-endian, the endianness 140*a9643ea8Slogwang setting doesn't change the endianness of the instructions on all 141*a9643ea8Slogwang architectures. That's why Itanium and SPARC filters work for both big 142*a9643ea8Slogwang and little endian executables (Itanium has little endian instructions 143*a9643ea8Slogwang and SPARC has big endian instructions). 144*a9643ea8Slogwang 145*a9643ea8Slogwang There currently is no filter for little endian PowerPC or big endian 146*a9643ea8Slogwang ARM or ARM-Thumb. Implementing filters for them can be considered if 147*a9643ea8Slogwang there is a need for such filters in real-world applications. 148*a9643ea8Slogwang 149*a9643ea8SlogwangNotes about shared libraries 150*a9643ea8Slogwang 151*a9643ea8Slogwang If you are including XZ Embedded into a shared library, you very 152*a9643ea8Slogwang probably should rename the xz_* functions to prevent symbol 153*a9643ea8Slogwang conflicts in case your library is linked against some other library 154*a9643ea8Slogwang or application that also has XZ Embedded in it (which may even be 155*a9643ea8Slogwang a different version of XZ Embedded). TODO: Provide an easy way 156*a9643ea8Slogwang to do this. 157*a9643ea8Slogwang 158*a9643ea8Slogwang Please don't create a shared library of XZ Embedded itself unless 159*a9643ea8Slogwang it is fine to rebuild everything depending on that shared library 160*a9643ea8Slogwang everytime you upgrade to a newer version of XZ Embedded. There are 161*a9643ea8Slogwang no API or ABI stability guarantees between different versions of 162*a9643ea8Slogwang XZ Embedded. 163*a9643ea8Slogwang 164