f2fs: keep POSIX_FADV_NOREUSE ranges

This patch records POSIX_FADV_NOREUSE ranges for users to reclaim the caches
instantly off from LRU.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaege

f2fs: keep POSIX_FADV_NOREUSE ranges

This patch records POSIX_FADV_NOREUSE ranges for users to reclaim the caches
instantly off from LRU.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: fix typos

Fix typos: datas -> data.

Via codespell.

Signed-off-by: Andrew Kreimer <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

f2fs: multidevice: add stats in debugfs

This patch adds per-device stats in debugfs, the examples
are as below:

mkfs.f2fs -f -c /dev/vdc /dev/vdb
mount /dev/vdb /mnt/f2fs/
cat /sys/kernel/debug/f2f

f2fs: multidevice: add stats in debugfs

This patch adds per-device stats in debugfs, the examples
are as below:

mkfs.f2fs -f -c /dev/vdc /dev/vdb
mount /dev/vdb /mnt/f2fs/
cat /sys/kernel/debug/f2fs/status

Multidevice stats:
[seg: inuse dirty full free prefree]
#0 5 0 0 4007 0
#1 1 0 0 8191 0

mkfs.f2fs -f -s 2 -c /dev/vdc /dev/vdb
mount /dev/vdb /mnt/f2fs
cat /sys/kernel/debug/f2fs/status

Multidevice stats:
[seg: inuse dirty full free prefree] [sec: inuse dirty full free prefree]
#0 5 0 0 4005 0 5 0 0 2000 0
#1 1 0 0 8191 0 1 0 0 4095 0

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: clean up val{>>,<<}F2FS_BLKSIZE_BITS

Use F2FS_BYTES_TO_BLK(bytes) and F2FS_BLK_TO_BYTES(blk) for cleanup

Signed-off-by: Zhiguo Niu <[email protected]>
Reviewed-by: Chao Yu <[email protected]

f2fs: clean up val{>>,<<}F2FS_BLKSIZE_BITS

Use F2FS_BYTES_TO_BLK(bytes) and F2FS_BLK_TO_BYTES(blk) for cleanup

Signed-off-by: Zhiguo Niu <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: introduce SEGS_TO_BLKS/BLKS_TO_SEGS for cleanup

Just cleanup, no functional change.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

f2fs: use BLKS_PER_SEG, BLKS_PER_SEC, and SEGS_PER_SEC

No functional change.

Reviewed-by: Daeho Jeong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <jaeg

f2fs: use BLKS_PER_SEG, BLKS_PER_SEC, and SEGS_PER_SEC

No functional change.

Reviewed-by: Daeho Jeong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: fix to account cp stats correctly

cp_foreground_calls sysfs entry shows total CP call count rather than
foreground CP call count, fix it.

Fixes: fc7100ea2a52 ("f2fs: Add f2fs stats to sysfs")

f2fs: fix to account cp stats correctly

cp_foreground_calls sysfs entry shows total CP call count rather than
foreground CP call count, fix it.

Fixes: fc7100ea2a52 ("f2fs: Add f2fs stats to sysfs")
Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: fix to account gc stats correctly

As reported, status debugfs entry shows inconsistent GC stats as below:

GC calls: 6008 (BG: 6161)
- data segments : 3053 (BG: 3053)
- node segments : 295

f2fs: fix to account gc stats correctly

As reported, status debugfs entry shows inconsistent GC stats as below:

GC calls: 6008 (BG: 6161)
- data segments : 3053 (BG: 3053)
- node segments : 2955 (BG: 2955)

Total GC calls is larger than BGGC calls, the reason is:
- f2fs_stat_info.call_count accounts total migrated section count
by f2fs_gc()
- f2fs_stat_info.bg_gc accounts total call times of f2fs_gc() from
background gc_thread

Another issue is gc_foreground_calls sysfs entry shows total GC call
count rather than FGGC call count.

This patch changes as below for fix:
- account GC calls and migrated segment count separately
- support to account migrated section count if it enables large section
mode
- fix to show correct value in gc_foreground_calls sysfs entry

Fixes: fc7100ea2a52 ("f2fs: Add f2fs stats to sysfs")
Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: fix to recover quota data correctly

With -O quota mkfs option, xfstests generic/417 fails due to fsck detects
data corruption on quota inodes.

[ASSERT] (fsck_chk_quota_files:2051) --> Quota

f2fs: fix to recover quota data correctly

With -O quota mkfs option, xfstests generic/417 fails due to fsck detects
data corruption on quota inodes.

[ASSERT] (fsck_chk_quota_files:2051) --> Quota file is missing or invalid quota file content found.

The root cause is there is a hole f2fs doesn't hold quota inodes,
so all recovered quota data will be dropped due to SBI_POR_DOING
flag was set.
- f2fs_fill_super
- f2fs_recover_orphan_inodes
- f2fs_enable_quota_files
- f2fs_quota_off_umount
<--- quota inodes were dropped --->
- f2fs_recover_fsync_data
- f2fs_enable_quota_files
- f2fs_quota_off_umount

This patch tries to eliminate the hole by holding quota inodes
during entire recovery flow as below:
- f2fs_fill_super
- f2fs_recover_quota_begin
- f2fs_recover_orphan_inodes
- f2fs_recover_fsync_data
- f2fs_recover_quota_end

Then, recovered quota data can be persisted after SBI_POR_DOING
is cleared.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: convert to MAX_SBI_FLAG instead of 32 in stat_show()

BIW reduce the s_flag array size and make s_flag constant.

Signed-off-by: Yangtao Li <[email protected]>
Reviewed-by: Chao Yu <chao@kernel

f2fs: convert to MAX_SBI_FLAG instead of 32 in stat_show()

BIW reduce the s_flag array size and make s_flag constant.

Signed-off-by: Yangtao Li <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: replace si->sbi w/ sbi in stat_show()

For each loop add a local f2fs_sb_info pointer insted of looking it up.

Signed-off-by: Yangtao Li <[email protected]>
Reviewed-by: Chao Yu <[email protected]

f2fs: replace si->sbi w/ sbi in stat_show()

For each loop add a local f2fs_sb_info pointer insted of looking it up.

Signed-off-by: Yangtao Li <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: export ipu policy in debugfs

Export ipu_policy as a string in debugfs for better readability and
it can help us better understand some strategies of the file system.

Signed-off-by: Yangtao Li

f2fs: export ipu policy in debugfs

Export ipu_policy as a string in debugfs for better readability and
it can help us better understand some strategies of the file system.

Signed-off-by: Yangtao Li <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: fix some format WARNING in debug.c and sysfs.c

To fix:

WARNING: function definition argument 'struct f2fs_attr *' should also have an identifier name
+ ssize_t (*show)(struct f2fs_attr

f2fs: fix some format WARNING in debug.c and sysfs.c

To fix:

WARNING: function definition argument 'struct f2fs_attr *' should also have an identifier name
+ ssize_t (*show)(struct f2fs_attr *, struct f2fs_sb_info *, char *);

WARNING: return sysfs_emit(...) formats should include a terminating newline
+ return sysfs_emit(buf, "(none)");

WARNING: Prefer 'unsigned int' to bare use of 'unsigned'
+ unsigned npages = NODE_MAPPING(sbi)->nrpages;

WARNING: Missing a blank line after declarations
+ unsigned npages = COMPRESS_MAPPING(sbi)->nrpages;
+ si->page_mem += (unsigned long long)npages << PAGE_SHIFT;

WARNING: quoted string split across lines
+ seq_printf(s, "CP merge (Queued: %4d, Issued: %4d, Total: %4d, "
+ "Cur time: %4d(ms), Peak time: %4d(ms))\n",

Signed-off-by: Yangtao Li <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garb

f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.

Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286

Signed-off-by: qixiaoyu1 <[email protected]>
Signed-off-by: xiongping1 <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: refactor extent_cache to support for read and more

This patch prepares extent_cache to be ready for addition.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel

f2fs: refactor extent_cache to support for read and more

This patch prepares extent_cache to be ready for addition.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: change to use atomic_t type form sbi.atomic_files

inode_lock[ATOMIC_FILE] was used for protecting sbi->atomic_files,
update atomic_files variable's type to atomic_t instead of unsigned
int, th

f2fs: change to use atomic_t type form sbi.atomic_files

inode_lock[ATOMIC_FILE] was used for protecting sbi->atomic_files,
update atomic_files variable's type to atomic_t instead of unsigned
int, then inode_lock[ATOMIC_FILE] can be obsoleted.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: account swapfile inodes

In order to check count of opened swapfile inodes.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

f2fs: code clean and fix a type error

ERROR: code indent should use tabs where possible
ERROR: spaces required around that ':'
ERROR: incorrect tab

Found serveral code type errors when review the c

f2fs: code clean and fix a type error

ERROR: code indent should use tabs where possible
ERROR: spaces required around that ':'
ERROR: incorrect tab

Found serveral code type errors when review the code and fix it.
There is no function change.

Signed-off-by: Zhang Qilong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: adjust zone capacity when considering valid block count

This patch fixes counting unusable blocks set by zone capacity when
checking the valid block count in a section.

Reviewed-by: Chao Yu <

f2fs: adjust zone capacity when considering valid block count

This patch fixes counting unusable blocks set by zone capacity when
checking the valid block count in a section.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: kill volatile write support

There's no user, since all can use atomic writes simply.
Let's kill it.

Signed-off-by: Jaegeuk Kim <[email protected]>

f2fs: change the current atomic write way

Current atomic write has three major issues like below.
- keeps the updates in non-reclaimable memory space and they are even
hard to be migrated, which

f2fs: change the current atomic write way

Current atomic write has three major issues like below.
- keeps the updates in non-reclaimable memory space and they are even
hard to be migrated, which is not good for contiguous memory
allocation.
- disk spaces used for atomic files cannot be garbage collected, so
this makes it difficult for the filesystem to be defragmented.
- If atomic write operations hit the threshold of either memory usage
or garbage collection failure count, All the atomic write operations
will fail immediately.

To resolve the issues, I will keep a COW inode internally for all the
updates to be flushed from memory, when we need to flush them out in a
situation like high memory pressure. These COW inodes will be tagged
as orphan inodes to be reclaimed in case of sudden power-cut or system
failure during atomic writes.

Signed-off-by: Daeho Jeong <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: introduce gc_urgent_mid mode

We need a mid level of gc urgent mode to do GC forcibly in a period
of given gc_urgent_sleep_time, but not like using greedy GC approach
and switching to SSR mode

f2fs: introduce gc_urgent_mid mode

We need a mid level of gc urgent mode to do GC forcibly in a period
of given gc_urgent_sleep_time, but not like using greedy GC approach
and switching to SSR mode such as gc urgent high mode. This can be
used for more aggressive periodic storage clean up.

Signed-off-by: Daeho Jeong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: use spin_lock to avoid hang

[14696.634553] task:cat state:D stack: 0 pid:1613738 ppid:1613735 flags:0x00000004
[14696.638285] Call Trace:
[14696.639038] <TASK>
[14696.640032]

f2fs: use spin_lock to avoid hang

[14696.634553] task:cat state:D stack: 0 pid:1613738 ppid:1613735 flags:0x00000004
[14696.638285] Call Trace:
[14696.639038] <TASK>
[14696.640032] __schedule+0x302/0x930
[14696.640969] schedule+0x58/0xd0
[14696.641799] schedule_preempt_disabled+0x18/0x30
[14696.642890] __mutex_lock.constprop.0+0x2fb/0x4f0
[14696.644035] ? mod_objcg_state+0x10c/0x310
[14696.645040] ? obj_cgroup_charge+0xe1/0x170
[14696.646067] __mutex_lock_slowpath+0x13/0x20
[14696.647126] mutex_lock+0x34/0x40
[14696.648070] stat_show+0x25/0x17c0 [f2fs]
[14696.649218] seq_read_iter+0x120/0x4b0
[14696.650289] ? aa_file_perm+0x12a/0x500
[14696.651357] ? lru_cache_add+0x1c/0x20
[14696.652470] seq_read+0xfd/0x140
[14696.653445] full_proxy_read+0x5c/0x80
[14696.654535] vfs_read+0xa0/0x1a0
[14696.655497] ksys_read+0x67/0xe0
[14696.656502] __x64_sys_read+0x1a/0x20
[14696.657580] do_syscall_64+0x3b/0xc0
[14696.658671] entry_SYSCALL_64_after_hwframe+0x44/0xae
[14696.660068] RIP: 0033:0x7efe39df1cb2
[14696.661133] RSP: 002b:00007ffc8badd948 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[14696.662958] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007efe39df1cb2
[14696.664757] RDX: 0000000000020000 RSI: 00007efe399df000 RDI: 0000000000000003
[14696.666542] RBP: 00007efe399df000 R08: 00007efe399de010 R09: 00007efe399de010
[14696.668363] R10: 0000000000000022 R11: 0000000000000246 R12: 0000000000000000
[14696.670155] R13: 0000000000000003 R14: 0000000000020000 R15: 0000000000020000
[14696.671965] </TASK>
[14696.672826] task:umount state:D stack: 0 pid:1614985 ppid:1614984 flags:0x00004000
[14696.674930] Call Trace:
[14696.675903] <TASK>
[14696.676780] __schedule+0x302/0x930
[14696.677927] schedule+0x58/0xd0
[14696.679019] schedule_preempt_disabled+0x18/0x30
[14696.680412] __mutex_lock.constprop.0+0x2fb/0x4f0
[14696.681783] ? destroy_inode+0x65/0x80
[14696.683006] __mutex_lock_slowpath+0x13/0x20
[14696.684305] mutex_lock+0x34/0x40
[14696.685442] f2fs_destroy_stats+0x1e/0x60 [f2fs]
[14696.686803] f2fs_put_super+0x158/0x390 [f2fs]
[14696.688238] generic_shutdown_super+0x7a/0x120
[14696.689621] kill_block_super+0x27/0x50
[14696.690894] kill_f2fs_super+0x7f/0x100 [f2fs]
[14696.692311] deactivate_locked_super+0x35/0xa0
[14696.693698] deactivate_super+0x40/0x50
[14696.694985] cleanup_mnt+0x139/0x190
[14696.696209] __cleanup_mnt+0x12/0x20
[14696.697390] task_work_run+0x64/0xa0
[14696.698587] exit_to_user_mode_prepare+0x1b7/0x1c0
[14696.700053] syscall_exit_to_user_mode+0x27/0x50
[14696.701418] do_syscall_64+0x48/0xc0
[14696.702630] entry_SYSCALL_64_after_hwframe+0x44/0xae

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: don't get FREEZE lock in f2fs_evict_inode in frozen fs

Let's purge inode cache in order to avoid the below deadlock.

[freeze test] shrinkder
freeze_super
- pwercpu_do

f2fs: don't get FREEZE lock in f2fs_evict_inode in frozen fs

Let's purge inode cache in order to avoid the below deadlock.

[freeze test] shrinkder
freeze_super
- pwercpu_down_write(SB_FREEZE_FS)
- super_cache_scan
- down_read(&sb->s_umount)
- prune_icache_sb
- dispose_list
- evict
- f2fs_evict_inode
thaw_super
- down_write(&sb->s_umount);
- __percpu_down_read(SB_FREEZE_FS)

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...

f2fs: add a way to limit roll forward recovery time

This adds a sysfs entry to call checkpoint during fsync() in order to avoid
long elapsed time to run roll-forward recovery when booting the device

f2fs: add a way to limit roll forward recovery time

This adds a sysfs entry to call checkpoint during fsync() in order to avoid
long elapsed time to run roll-forward recovery when booting the device.
Default value doesn't enforce the limitation which is same as before.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

show more ...