squashfs: update Kconfig information

Update the compression algorithms supported, and the Squashfs website
location.

Link: https://lkml.kernel.org/r/[email protected]
S

squashfs: update Kconfig information

Update the compression algorithms supported, and the Squashfs website
location.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Phillip Lougher <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

squashfs: allows users to configure the number of decompression threads

The maximum number of threads in the decompressor_multi.c file is fixed
and cannot be adjusted according to user needs. There

squashfs: allows users to configure the number of decompression threads

The maximum number of threads in the decompressor_multi.c file is fixed
and cannot be adjusted according to user needs. Therefore, the mount
parameter needs to be added to allow users to configure the number of
threads as required. The upper limit is num_online_cpus() * 2.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Xiaoming Ni <[email protected]>
Reviewed-by: Phillip Lougher <[email protected]>
Cc: Jianguo Chen <[email protected]>
Cc: Jubin Zhong <[email protected]>
Cc: Zhang Yi <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

squashfs: add the mount parameter theads=<single|multi|percpu>

Patch series 'squashfs: Add the mount parameter "threads="'.

Currently, Squashfs supports multiple decompressor parallel modes.
Howev

squashfs: add the mount parameter theads=<single|multi|percpu>

Patch series 'squashfs: Add the mount parameter "threads="'.

Currently, Squashfs supports multiple decompressor parallel modes.
However, this mode can be configured only during kernel building and does
not support flexible selection during runtime.

In the current patch set, the mount parameter "threads=" is added to allow
users to select the parallel decompressor mode and configure the number of
decompressors when mounting a file system.

"threads=<single|multi|percpu|1|2|3|...>"
The upper limit is num_online_cpus() * 2.


This patch (of 2):

Squashfs supports three decompression concurrency modes:
Single-thread mode: concurrent reads are blocked and the memory
overhead is small.
Multi-thread mode/percpu mode: reduces concurrent read blocking but
increases memory overhead.

The corresponding schema must be fixed at compile time. During mounting,
the concurrent decompression mode cannot be adjusted based on file read
blocking.

The mount parameter theads=<single|multi|percpu> is added to select
the concurrent decompression mode of a single SquashFS file system
image.

Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Xiaoming Ni <[email protected]>
Reviewed-by: Phillip Lougher <[email protected]>
Cc: Jianguo Chen <[email protected]>
Cc: Jubin Zhong <[email protected]>
Cc: Zhang Yi <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

treewide: Add SPDX license identifier - Makefile/Kconfig

Add SPDX license identifiers to all Make/Kconfig files which:

- Have no license information of any form

These files fall under the project

treewide: Add SPDX license identifier - Makefile/Kconfig

Add SPDX license identifiers to all Make/Kconfig files which:

- Have no license information of any form

These files fall under the project license, GPL v2 only. The resulting SPDX
license identifier is:

GPL-2.0-only

Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

squashfs: Add zstd support

Add zstd compression and decompression support to SquashFS. zstd is a
great fit for SquashFS because it can compress at ratios approaching xz,
while decompressing twice as

squashfs: Add zstd support

Add zstd compression and decompression support to SquashFS. zstd is a
great fit for SquashFS because it can compress at ratios approaching xz,
while decompressing twice as fast as zlib. For SquashFS in particular,
it can decompress as fast as lzo and lz4. It also has the flexibility
to turn down the compression ratio for faster compression times.

The compression benchmark is run on the file tree from the SquashFS archive
found in ubuntu-16.10-desktop-amd64.iso [1]. It uses `mksquashfs` with the
default block size (128 KB) and and various compression algorithms/levels.
xz and zstd are also benchmarked with 256 KB blocks. The decompression
benchmark times how long it takes to `tar` the file tree into `/dev/null`.
See the benchmark file in the upstream zstd source repository located under
`contrib/linux-kernel/squashfs-benchmark.sh` [2] for details.

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD.

| Method | Ratio | Compression MB/s | Decompression MB/s |
|----------------|-------|------------------|--------------------|
| gzip | 2.92 | 15 | 128 |
| lzo | 2.64 | 9.5 | 217 |
| lz4 | 2.12 | 94 | 218 |
| xz | 3.43 | 5.5 | 35 |
| xz 256 KB | 3.53 | 5.4 | 40 |
| zstd 1 | 2.71 | 96 | 210 |
| zstd 5 | 2.93 | 69 | 198 |
| zstd 10 | 3.01 | 41 | 225 |
| zstd 15 | 3.13 | 11.4 | 224 |
| zstd 16 256 KB | 3.24 | 8.1 | 210 |

This patch was written by Sean Purcell <[email protected]>, but I will be
taking over the submission process.

[1] http://releases.ubuntu.com/16.10/
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/squashfs-benchmark.sh

zstd source repository: https://github.com/facebook/zstd

Signed-off-by: Sean Purcell <[email protected]>
Signed-off-by: Nick Terrell <[email protected]>
Signed-off-by: Chris Mason <[email protected]>
Acked-by: Phillip Lougher <[email protected]>

show more ...

Squashfs: Add LZ4 compression configuration option

Add the glue code, and also update the documentation.

Signed-off-by: Phillip Lougher <[email protected]>

Squashfs: Directly decompress into the page cache for file data

This introduces an implementation of squashfs_readpage_block()
that directly decompresses into the page cache.

This uses the previous

Squashfs: Directly decompress into the page cache for file data

This introduces an implementation of squashfs_readpage_block()
that directly decompresses into the page cache.

This uses the previously added page handler abstraction to push
down the necessary kmap_atomic/kunmap_atomic operations on the
page cache buffers into the decompressors. This enables
direct copying into the page cache without using the slow
kmap/kunmap calls.

The code detects when multiple threads are racing in
squashfs_readpage() to decompress the same block, and avoids
this regression by falling back to using an intermediate
buffer.

This patch enhances the performance of Squashfs significantly
when multiple processes are accessing the filesystem simultaneously
because it not only reduces memcopying, but it more importantly
eliminates the lock contention on the intermediate buffer.

Using single-thread decompression.

dd if=file1 of=/dev/null bs=4096 &
dd if=file2 of=/dev/null bs=4096 &
dd if=file3 of=/dev/null bs=4096 &
dd if=file4 of=/dev/null bs=4096

Before:

629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s

After:

629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s

Signed-off-by: Phillip Lougher <[email protected]>
Reviewed-by: Minchan Kim <[email protected]>

show more ...

Squashfs: add multi-threaded decompression using percpu variable

Add a multi-threaded decompression implementation which uses
percpu variables.

Using percpu variables has advantages and disadvantag

Squashfs: add multi-threaded decompression using percpu variable

Add a multi-threaded decompression implementation which uses
percpu variables.

Using percpu variables has advantages and disadvantages over
implementations which do not use percpu variables.

Advantages:
* the nature of percpu variables ensures decompression is
load-balanced across the multiple cores.
* simplicity.

Disadvantages: it limits decompression to one thread per core.

Signed-off-by: Phillip Lougher <[email protected]>

show more ...

squashfs: Enhance parallel I/O

Now squashfs have used for only one stream buffer for decompression
so it hurts parallel read performance so this patch supports
multiple decompressor to enhance perfo

squashfs: Enhance parallel I/O

Now squashfs have used for only one stream buffer for decompression
so it hurts parallel read performance so this patch supports
multiple decompressor to enhance performance parallel I/O.

Four 1G file dd read on KVM machine which has 2 CPU and 4G memory.

dd if=test/test1.dat of=/dev/null &
dd if=test/test2.dat of=/dev/null &
dd if=test/test3.dat of=/dev/null &
dd if=test/test4.dat of=/dev/null &

old : 1m39s -> new : 9s

* From v1
* Change comp_strm with decomp_strm - Phillip
* Change/add comments - Phillip

Signed-off-by: Minchan Kim <[email protected]>
Signed-off-by: Phillip Lougher <[email protected]>

show more ...

Squashfs: Add an option to set dev block size to 4K

This commit adds an option to set the device block size used to 4K.

By default Squashfs sets the device block size (sb_min_blocksize) to 1K
or th

Squashfs: Add an option to set dev block size to 4K

This commit adds an option to set the device block size used to 4K.

By default Squashfs sets the device block size (sb_min_blocksize) to 1K
or the smallest block size supported by the block device (if larger).
This, because blocks are packed together and unaligned in Squashfs,
should reduce latency.

This, however, gives poor performance on MTD NAND devices where
the optimal I/O size is 4K (even though the devices can support
smaller block sizes).

Using a 4K device block size may also improve overall I/O
performance for some file access patterns (e.g. sequential
accesses of files in filesystem order) on all media.

Signed-off-by: Phillip Lougher <[email protected]>

show more ...

doc: fix broken references

There are numerous broken references to Documentation files (in other
Documentation files, in comments, etc.). These broken references are
caused by typo's in the referenc

doc: fix broken references

There are numerous broken references to Documentation files (in other
Documentation files, in comments, etc.). These broken references are
caused by typo's in the references, and by renames or removals of the
Documentation files. Some broken references are simply odd.

Fix these broken references, sometimes by dropping the irrelevant text
they were part of.

Signed-off-by: Paul Bolle <[email protected]>
Signed-off-by: Jiri Kosina <[email protected]>

show more ...

Squashfs: Make ZLIB compression support optional

Squashfs now supports XZ and LZO compression in addition to ZLIB.
As such it no longer makes sense to always include ZLIB support.
In particular embe

Squashfs: Make ZLIB compression support optional

Squashfs now supports XZ and LZO compression in addition to ZLIB.
As such it no longer makes sense to always include ZLIB support.
In particular embedded systems may only use LZO or XZ compression, and
the ability to exclude ZLIB support will reduce kernel size.

Signed-off-by: Phillip Lougher <[email protected]>

show more ...

treewide: fix a few typos in comments

- kenrel -> kernel
- whetehr -> whether
- ttt -> tt
- sss -> ss

Signed-off-by: Justin P. Mattock <[email protected]>
Signed-off-by: Jiri Kosina <jkosina@

treewide: fix a few typos in comments

- kenrel -> kernel
- whetehr -> whether
- ttt -> tt
- sss -> ss

Signed-off-by: Justin P. Mattock <[email protected]>
Signed-off-by: Jiri Kosina <[email protected]>

show more ...

Squashfs: Update Kconfig help text to include xz compression

Signed-off-by: Phillip Lougher <[email protected]>

Squashfs: get rid of default n in Kconfig

As pointed out by Geert Uytterhoeven, "default n" is the default,
no reason to specify it.

Signed-off-by: Phillip Lougher <[email protected]>

Squashfs: Add XZ compression configuration option

Signed-off-by: Phillip Lougher <[email protected]>

Squashfs: update Kconfig and documentation for LZO

Update compression types supported and add some help text for
the LZO Kconfig option.

Also add missing "default n" line and make some trivial whit

Squashfs: update Kconfig and documentation for LZO

Update compression types supported and add some help text for
the LZO Kconfig option.

Also add missing "default n" line and make some trivial whitespace
cleanups too.

Signed-off-by: Phillip Lougher <[email protected]>

show more ...

Squashfs: Add LZO compression support

Signed-off-by: Chan Jeong <[email protected]>
Signed-off-by: Phillip Lougher <[email protected]>

Squashfs: Make XATTR config name consistent with other file systems

Reported-by: Geert Uytterhoeven <[email protected]>
Signed-off-by: Phillip Lougher <[email protected]>

squashfs: add xattr support configure option

Signed-off-by: Phillip Lougher <[email protected]>

fs/Kconfig: move squashfs out

Signed-off-by: Alexey Dobriyan <[email protected]>