fs: prepare for stackable filesystems backing file helpers

In preparation for factoring out some backing file io helpers from
overlayfs, move backing_file_open() into a new file fs/backing-file.c
an

fs: prepare for stackable filesystems backing file helpers

In preparation for factoring out some backing file io helpers from
overlayfs, move backing_file_open() into a new file fs/backing-file.c
and header.

Add a MAINTAINERS entry for stackable filesystems and add a Kconfig
FS_STACK which stackable filesystems need to select.

For now, the backing_file struct, the backing_file alloc/free functions
and the backing_file_real_path() accessor remain internal to file_table.c.
We may change that in the future.

Signed-off-by: Amir Goldstein <[email protected]>

show more ...

ovl: Kconfig: introduce CONFIG_OVERLAY_FS_DEBUG

Provide a Kconfig option to enable extra debugging checks for overlayfs.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Andrea Righi

ovl: Kconfig: introduce CONFIG_OVERLAY_FS_DEBUG

Provide a Kconfig option to enable extra debugging checks for overlayfs.

Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Andrea Righi <[email protected]>
Signed-off-by: Amir Goldstein <[email protected]>

show more ...

ovl: Kconfig: Fix spelling mistake "undelying" -> "underlying"

There is a spelling mistake in a Kconfig description. Fix it.

Signed-off-by: Colin Ian King <[email protected]>
Signed-off-by: Mi

ovl: Kconfig: Fix spelling mistake "undelying" -> "underlying"

There is a spelling mistake in a Kconfig description. Fix it.

Signed-off-by: Colin Ian King <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

docs: fix broken references to text files

Several references got broken due to txt to ReST conversion.

Several of them can be automatically fixed with:

scripts/documentation-file-ref-check --fix

docs: fix broken references to text files

Several references got broken due to txt to ReST conversion.

Several of them can be automatically fixed with:

scripts/documentation-file-ref-check --fix

Reviewed-by: Mathieu Poirier <[email protected]> # hwtracing/coresight/Kconfig
Reviewed-by: Paul E. McKenney <[email protected]> # memory-barrier.txt
Acked-by: Alex Shi <[email protected]> # translations/zh_CN
Acked-by: Federico Vaga <[email protected]> # translations/it_IT
Acked-by: Marc Zyngier <[email protected]> # kvm/arm64
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Link: https://lore.kernel.org/r/6f919ddb83a33b5f2a63b6b5f0575737bb2b36aa.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <[email protected]>

show more ...

ovl: fix some xino configurations

Fix up two bugs in the coversion to xino_mode:
1. xino=off does not always end up in disabled mode
2. xino=auto on 32bit arch should end up in disabled mode

Take a

ovl: fix some xino configurations

Fix up two bugs in the coversion to xino_mode:
1. xino=off does not always end up in disabled mode
2. xino=auto on 32bit arch should end up in disabled mode

Take a proactive approach to disabling xino on 32bit kernel:
1. Disable XINO_AUTO config during build time
2. Disable xino with a warning on mount time

As a by product, xino=on on 32bit arch also ends up in disabled mode.
We never intended to enable xino on 32bit arch and this will make the
rest of the logic simpler.

Fixes: 0f831ec85eda ("ovl: simplify ovl_same_sb() helper")
Signed-off-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

treewide: Add SPDX license identifier - Makefile/Kconfig

Add SPDX license identifiers to all Make/Kconfig files which:

- Have no license information of any form

These files fall under the project

treewide: Add SPDX license identifier - Makefile/Kconfig

Add SPDX license identifiers to all Make/Kconfig files which:

- Have no license information of any form

These files fall under the project license, GPL v2 only. The resulting SPDX
license identifier is:

GPL-2.0-only

Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

ovl: Provide a mount option metacopy=on/off for metadata copyup

By default metadata only copy up is disabled. Provide a mount option so
that users can choose one way or other.

Also provide a kerne

ovl: Provide a mount option metacopy=on/off for metadata copyup

By default metadata only copy up is disabled. Provide a mount option so
that users can choose one way or other.

Also provide a kernel config and module option to enable/disable metacopy
feature.

metacopy feature requires redirect_dir=on when upper is present.
Otherwise, it requires redirect_dir=follow atleast.

As of now, metacopy does not work with nfs_export=on. So if both
metacopy=on and nfs_export=on then nfs_export is disabled.

Signed-off-by: Vivek Goyal <[email protected]>
Reviewed-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: Kconfig documentation fixes

Reported-by: Randy Dunlap <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

ovl: add support for "xino" mount and config options

With mount option "xino=on", mounter declares that there are enough
free high bits in underlying fs to hold the layer fsid.
If overlayfs does enc

ovl: add support for "xino" mount and config options

With mount option "xino=on", mounter declares that there are enough
free high bits in underlying fs to hold the layer fsid.
If overlayfs does encounter underlying inodes using the high xino
bits reserved for layer fsid, a warning will be emitted and the original
inode number will be used.

The mount option name "xino" goes after a similar meaning mount option
of aufs, but in overlayfs case, the mapping is stateless.

An example for a use case of "xino=on" is when upper/lower is on an xfs
filesystem. xfs uses 64bit inode numbers, but it currently never uses the
upper 8bit for inode numbers exposed via stat(2) and that is not likely to
change in the future without user opting-in for a new xfs feature. The
actual number of unused upper bit is much larger and determined by the xfs
filesystem geometry (64 - agno_log - agblklog - inopblog). That means
that for all practical purpose, there are enough unused bits in xfs
inode numbers for more than OVL_MAX_STACK unique fsid's.

Another use case of "xino=on" is when upper/lower is on tmpfs. tmpfs inode
numbers are allocated sequentially since boot, so they will practially
never use the high inode number bits.

For compatibility with applications that expect 32bit inodes, the feature
can be disabled with "xino=off". The option "xino=auto" automatically
detects underlying filesystem that use 32bit inodes and enables the
feature. The Kconfig option OVERLAY_FS_XINO_AUTO and module parameter of
the same name, determine if the default mode for overlayfs mount is
"xino=auto" or "xino=off".

Signed-off-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: update Kconfig texts

Add some hints about overlayfs kernel config options.

Enabling NFS export by default is especially recommended against, as it
incurs a performance penalty even if the file

ovl: update Kconfig texts

Add some hints about overlayfs kernel config options.

Enabling NFS export by default is especially recommended against, as it
incurs a performance penalty even if the filesystem is not actually
exported.

Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: add support for "nfs_export" configuration

Introduce the "nfs_export" config, module and mount options.

The NFS export feature depends on the "index" feature and enables two
implicit overlayfs

ovl: add support for "nfs_export" configuration

Introduce the "nfs_export" config, module and mount options.

The NFS export feature depends on the "index" feature and enables two
implicit overlayfs features: "index_all" and "verify_lower".
The "index_all" feature creates an index on copy up of every file and
directory. The "verify_lower" feature uses the full index to detect
overlay filesystems inconsistencies on lookup, like redirect from
multiple upper dirs to the same lower dir.

NFS export can be enabled for non-upper mount with no index. However,
because lower layer redirects cannot be verified with the index, enabling
NFS export support on an overlay with no upper layer requires turning off
redirect follow (e.g. "redirect_dir=nofollow").

The full index may incur some overhead on mount time, especially when
verifying that lower directory file handles are not stale.

NFS export support, full index and consistency verification will be
implemented by following patches.

Signed-off-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: update documentation of inodes index feature

Document that inode index feature solves breaking hard links on
copy up.

Simplify Kconfig backward compatibility disclaimer.

Signed-off-by: Amir G

ovl: update documentation of inodes index feature

Document that inode index feature solves breaking hard links on
copy up.

Simplify Kconfig backward compatibility disclaimer.

Signed-off-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: don't follow redirects if redirect_dir=off

Overlayfs is following redirects even when redirects are disabled. If this
is unintentional (probably the majority of cases) then this can be a
proble

ovl: don't follow redirects if redirect_dir=off

Overlayfs is following redirects even when redirects are disabled. If this
is unintentional (probably the majority of cases) then this can be a
problem. E.g. upper layer comes from untrusted USB drive, and attacker
crafts a redirect to enable read access to otherwise unreadable
directories.

If "redirect_dir=off", then turn off following as well as creation of
redirects. If "redirect_dir=follow", then turn on following, but turn off
creation of redirects (which is what "redirect_dir=off" does now).

This is a backward incompatible change, so make it dependent on a config
option.

Reported-by: David Howells <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: introduce the inodes index dir feature

Create the index dir on mount. The index dir will contain hardlinks to
upper inodes, named after the hex representation of their origin lower
inodes.

The

ovl: introduce the inodes index dir feature

Create the index dir on mount. The index dir will contain hardlinks to
upper inodes, named after the hex representation of their origin lower
inodes.

The index dir is going to be used to prevent breaking lower hardlinks
on copy up and to implement overlayfs NFS export.

Because the feature is not fully backward compat, enabling the feature
is opt-in by config/module/mount option.

Signed-off-by: Amir Goldstein <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: select EXPORTFS

We get a link error when EXPORTFS is not enabled:

ERROR: "exportfs_encode_fh" [fs/overlayfs/overlay.ko] undefined!
ERROR: "exportfs_decode_fh" [fs/overlayfs/overlay.ko] undefin

ovl: select EXPORTFS

We get a link error when EXPORTFS is not enabled:

ERROR: "exportfs_encode_fh" [fs/overlayfs/overlay.ko] undefined!
ERROR: "exportfs_decode_fh" [fs/overlayfs/overlay.ko] undefined!

This adds a Kconfig 'select' statement for overlayfs, the same way that
it is done for the other users of exportfs.

Fixes: 3a1e819b4e80 ("ovl: store file handle of lower inode on copy up")
Signed-off-by: Arnd Bergmann <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: allow redirect_dir to default to "on"

This patch introduces a kernel config option and a module param. Both can
be used independently to turn the default value of redirect_dir on or off.

Sign

ovl: allow redirect_dir to default to "on"

This patch introduces a kernel config option and a module param. Both can
be used independently to turn the default value of redirect_dir on or off.

Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

ovl: rename filesystem type to "overlay"

Some distributions carry an "old" format of overlayfs while mainline has a
"new" format.

The distros will possibly want to keep the old overlayfs alongside

ovl: rename filesystem type to "overlay"

Some distributions carry an "old" format of overlayfs while mainline has a
"new" format.

The distros will possibly want to keep the old overlayfs alongside the new
for compatibility reasons.

To make it possible to differentiate the two versions change the name of
the new one from "overlayfs" to "overlay".

Signed-off-by: Miklos Szeredi <[email protected]>
Reported-by: Serge Hallyn <[email protected]>
Cc: Andy Whitcroft <[email protected]>

show more ...

overlay filesystem

Overlayfs allows one, usually read-write, directory tree to be
overlaid onto another, read-only directory tree. All modifications
go to the upper, writable layer.

This type of m

overlay filesystem

Overlayfs allows one, usually read-write, directory tree to be
overlaid onto another, read-only directory tree. All modifications
go to the upper, writable layer.

This type of mechanism is most often used for live CDs but there's a
wide variety of other uses.

The implementation differs from other "union filesystem"
implementations in that after a file is opened all operations go
directly to the underlying, lower or upper, filesystems. This
simplifies the implementation and allows native performance in these
cases.

The dentry tree is duplicated from the underlying filesystems, this
enables fast cached lookups without adding special support into the
VFS. This uses slightly more memory than union mounts, but dentries
are relatively small.

Currently inodes are duplicated as well, but it is a possible
optimization to share inodes for non-directories.

Opening non directories results in the open forwarded to the
underlying filesystem. This makes the behavior very similar to union
mounts (with the same limitations vs. fchmod/fchown on O_RDONLY file
descriptors).

Usage:

mount -t overlayfs overlayfs -olowerdir=/lower,upperdir=/upper/upper,workdir=/upper/work /overlay

The following cotributions have been folded into this patch:

Neil Brown <[email protected]>:
- minimal remount support
- use correct seek function for directories
- initialise is_real before use
- rename ovl_fill_cache to ovl_dir_read

Felix Fietkau <[email protected]>:
- fix a deadlock in ovl_dir_read_merged
- fix a deadlock in ovl_remove_whiteouts

Erez Zadok <[email protected]>
- fix cleanup after WARN_ON

Sedat Dilek <[email protected]>
- fix up permission to confirm to new API

Robin Dong <[email protected]>
- fix possible leak in ovl_new_inode
- create new inode in ovl_link

Andy Whitcroft <[email protected]>
- switch to __inode_permission()
- copy up i_uid/i_gid from the underlying inode

AV:
- ovl_copy_up_locked() - dput(ERR_PTR(...)) on two failure exits
- ovl_clear_empty() - one failure exit forgetting to do unlock_rename(),
lack of check for udir being the parent of upper, dropping and regaining
the lock on udir (which would require _another_ check for parent being
right).
- bogus d_drop() in copyup and rename [fix from your mail]
- copyup/remove and copyup/rename races [fix from your mail]
- ovl_dir_fsync() leaving ERR_PTR() in ->realfile
- ovl_entry_free() is pointless - it's just a kfree_rcu()
- fold ovl_do_lookup() into ovl_lookup()
- manually assigning ->d_op is wrong. Just use ->s_d_op.
[patches picked from Miklos]:
* copyup/remove and copyup/rename races
* bogus d_drop() in copyup and rename

Also thanks to the following people for testing and reporting bugs:

Jordi Pujol <[email protected]>
Andy Whitcroft <[email protected]>
Michal Suchanek <[email protected]>
Felix Fietkau <[email protected]>
Erez Zadok <[email protected]>
Randy Dunlap <[email protected]>

Signed-off-by: Miklos Szeredi <[email protected]>

show more ...