fs/buffer: optimize discard_buffer()

While invalidating, the clearing of the bits in discard_buffer()
is done in one fully ordered CAS operation. In the past this was
done via individual clear_bit()

fs/buffer: optimize discard_buffer()

While invalidating, the clearing of the bits in discard_buffer()
is done in one fully ordered CAS operation. In the past this was
done via individual clear_bit(), until e7470ee89f0 (fs: buffer:
do not use unnecessary atomic operations when discarding buffers).
This implies that there were never strong ordering requirements
outside of being serialized by the buffer lock.

As such relax the ordering for archs that can benefit. Further,
the implied ordering in buffer_unlock() makes current cmpxchg
implied barrier redundant due to release semantics. And while in
theory the unlock could be part of the bulk clearing, it is
best to leave it explicit, but without the double barriers.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: remove superfluous statements

Get rid of those unnecessary return statements.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/20250515173925.147823-4-dav

fs/buffer: remove superfluous statements

Get rid of those unnecessary return statements.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: avoid redundant lookup in getblk slowpath

__getblk_slow() already implies failing a first lookup
as the fastpath, so try to create the buffers immediately
and avoid the redundant lookup.

fs/buffer: avoid redundant lookup in getblk slowpath

__getblk_slow() already implies failing a first lookup
as the fastpath, so try to create the buffers immediately
and avoid the redundant lookup. This saves 5-10% of the
total cost/latency of the slowpath.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: use sleeping lookup in __getblk_slowpath()

Just as with the fast path, call the lookup variant depending
on the gfp flags.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https:

fs/buffer: use sleeping lookup in __getblk_slowpath()

Just as with the fast path, call the lookup variant depending
on the gfp flags.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: Remove redundant errseq_set call in mark_buffer_write_io_error.

mark_buffer_write_io_error sets sb->s_wb_err to -EIO twice.
Once in mapping_set_error and once in errseq_set.
Only mapping_set_err

fs: Remove redundant errseq_set call in mark_buffer_write_io_error.

mark_buffer_write_io_error sets sb->s_wb_err to -EIO twice.
Once in mapping_set_error and once in errseq_set.
Only mapping_set_error checks if bh->b_assoc_map->host is NULL.

Discovered during null pointer dereference during writeback
to a failing device:

[<ffffffff9a416dc8>] ? mark_buffer_write_io_error+0x98/0xc0
[<ffffffff9a416dbe>] ? mark_buffer_write_io_error+0x8e/0xc0
[<ffffffff9ad4bda0>] end_buffer_async_write+0x90/0xd0
[<ffffffff9ad4e3eb>] end_bio_bh_io_sync+0x2b/0x40
[<ffffffff9adbafe6>] blk_update_request+0x1b6/0x480
[<ffffffff9adbb3d8>] blk_mq_end_request+0x18/0x30
[<ffffffff9adbc6aa>] blk_mq_dispatch_rq_list+0x4da/0x8e0
[<ffffffff9adc0a68>] __blk_mq_sched_dispatch_requests+0x218/0x6a0
[<ffffffff9adc07fa>] blk_mq_sched_dispatch_requests+0x3a/0x80
[<ffffffff9adbbb98>] blk_mq_run_hw_queue+0x108/0x330
[<ffffffff9adbcf58>] blk_mq_flush_plug_list+0x178/0x5f0
[<ffffffff9adb6741>] __blk_flush_plug+0x41/0x120
[<ffffffff9adb6852>] blk_finish_plug+0x22/0x40
[<ffffffff9ad47cb0>] wb_writeback+0x150/0x280
[<ffffffff9ac5343f>] ? set_worker_desc+0x9f/0xc0
[<ffffffff9ad4676e>] wb_workfn+0x24e/0x4a0

Fixes: 485e9605c0573 ("fs/buffer.c: record blockdev write errors in super_block that it backs")
Signed-off-by: Jeremy Bongio <[email protected]>
Link: https://lore.kernel.org/[email protected]
Reviewed-by: Christoph Hellwig <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

mm/migrate: fix sleep in atomic for large folios and buffer heads

The large folio + buffer head noref migration scenarios are
being naughty and blocking while holding a spinlock.

As a consequence o

mm/migrate: fix sleep in atomic for large folios and buffer heads

The large folio + buffer head noref migration scenarios are
being naughty and blocking while holding a spinlock.

As a consequence of the pagecache lookup path taking the
folio lock this serializes against migration paths, so
they can wait for each other. For the private_lock
atomic case, a new BH_Migrate flag is introduced which
enables the lookup to bail.

This allows the critical region of the private_lock on
the migration path to be reduced to the way it was before
ebdf4de5642fb6 ("mm: migrate: fix reference check race
between __find_get_block() and migration"), that is covering
the count checks.

The scope is always noref migration.

Reported-by: kernel test robot <[email protected]>
Reported-by: [email protected]
Closes: https://lore.kernel.org/oe-lkp/[email protected]
Fixes: 3c20917120ce61 ("block/bdev: enable large folio support for large logical block sizes")
Reviewed-by: Jan Kara <[email protected]>
Co-developed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://kdevops.org/ext4/v6.15-rc2.html # [0]
Link: https://lore.kernel.org/all/[email protected]/ # [1]
Link: https://lore.kernel.org/[email protected]
Tested-by: [email protected] # [0] [1]
Reviewed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: use sleeping version of __find_get_block()

Convert to the new nonatomic flavor to benefit from potential performance
benefits and adapt in the future vs migration such that semantics
are

fs/buffer: use sleeping version of __find_get_block()

Convert to the new nonatomic flavor to benefit from potential performance
benefits and adapt in the future vs migration such that semantics
are kept.

Convert write_boundary_block() which already takes the buffer
lock as well as bdev_getblk() depending on the respective gpf flags.
There are no changes in semantics.

Suggested-by: Jan Kara <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://kdevops.org/ext4/v6.15-rc2.html # [0]
Link: https://lore.kernel.org/all/[email protected]/ # [1]
Link: https://lore.kernel.org/[email protected]
Tested-by: [email protected] # [0] [1]
Reviewed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: introduce sleeping flavors for pagecache lookups

Add __find_get_block_nonatomic() and sb_find_get_block_nonatomic()
calls for which users will be converted where safe. These versions
will

fs/buffer: introduce sleeping flavors for pagecache lookups

Add __find_get_block_nonatomic() and sb_find_get_block_nonatomic()
calls for which users will be converted where safe. These versions
will take the folio lock instead of the mapping's private_lock.

Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://kdevops.org/ext4/v6.15-rc2.html # [0]
Link: https://lore.kernel.org/all/[email protected]/ # [1]
Link: https://lore.kernel.org/[email protected]
Tested-by: [email protected]
Reviewed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: split locking for pagecache lookups

Callers of __find_get_block() may or may not allow for blocking
semantics, and is currently assumed that it will not. Layout
two paths based on this. T

fs/buffer: split locking for pagecache lookups

Callers of __find_get_block() may or may not allow for blocking
semantics, and is currently assumed that it will not. Layout
two paths based on this. The the private_lock scheme will
continued to be used for atomic contexts. Otherwise take the
folio lock instead, which protects the buffers, such as
vs migration and try_to_free_buffers().

Per the "hack idea", the latter can alleviate contention on
the private_lock for bdev mappings. For reasons of determinism
and avoid making bugs hard to reproduce, the trylocking is not
attempted.

No change in semantics. All lookup users still take the spinlock.

Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://kdevops.org/ext4/v6.15-rc2.html # [0]
Link: https://lore.kernel.org/all/[email protected]/ # [1]
Link: https://lore.kernel.org/[email protected]
Tested-by: [email protected]
Reviewed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: convert block_commit_write() to take a folio

All callers now have a folio, so pass it in instead of converting
folio->page->folio.

Link: https://lkml.kernel.org/r/20250217192009.437916-1-willy@

fs: convert block_commit_write() to take a folio

All callers now have a folio, so pass it in instead of converting
folio->page->folio.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs/buffer fs/mpage: remove large folio restriction

Now that buffer-heads has been converted over to support large folios
we can remove the built-in VM_BUG_ON_FOLIO() checks which prevents
their use.

fs/buffer fs/mpage: remove large folio restriction

Now that buffer-heads has been converted over to support large folios
we can remove the built-in VM_BUG_ON_FOLIO() checks which prevents
their use.

Reviewed-by: Hannes Reinecke <[email protected]>
Reviewed-by: "Matthew Wilcox (Oracle)" <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: John Garry <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: remove batching from async read

block_read_full_folio() currently puts all !uptodate buffers into
an array allocated on the stack, then iterates over it twice, first
locking the buffers a

fs/buffer: remove batching from async read

block_read_full_folio() currently puts all !uptodate buffers into
an array allocated on the stack, then iterates over it twice, first
locking the buffers and then submitting them for read. We want to
remove this array because it occupies too much stack space on
configurations with a larger PAGE_SIZE (eg 512 bytes with 8 byte
pointers and a 64KiB PAGE_SIZE).

We cannot simply submit buffer heads as we find them as the completion
handler needs to be able to tell when all reads are finished, so it can
end the folio read. So we keep one buffer in reserve (using the 'prev'
variable) until the end of the function.

Reviewed-by: Hannes Reinecke <[email protected]>
Signed-off-by: "Matthew Wilcox (Oracle)" <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/buffer: simplify block_read_full_folio() with bh_offset()

When we read over all buffers in a folio we currently use the
buffer index on the folio and blocksize to get the offset. Simplify
this wi

fs/buffer: simplify block_read_full_folio() with bh_offset()

When we read over all buffers in a folio we currently use the
buffer index on the folio and blocksize to get the offset. Simplify
this with bh_offset(). This simplifies the loop while making no
functional changes.

Suggested-by: Matthew Wilcox <[email protected]>
Reviewed-by: Hannes Reinecke <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Christian Brauner <[email protected]>

show more ...

memcg-v1: no need for memcg locking for dirty tracking

During the era of memcg charge migration, the kernel has to be make
sure that the dirty stat updates do not race with the charge migration.
Oth

memcg-v1: no need for memcg locking for dirty tracking

During the era of memcg charge migration, the kernel has to be make
sure that the dirty stat updates do not race with the charge migration.
Otherwise it might update the dirty stats of the wrong memcg. Now
with the memcg charge migration gone, there is no more race for dirty
stat updates and the previous locking can be removed.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Shakeel Butt <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Reviewed-by: Roman Gushchin <[email protected]>
Acked-by: Johannes Weiner <[email protected]>
Cc: Hugh Dickins <[email protected]>
Cc: Muchun Song <[email protected]>
Cc: Yosry Ahmed <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs/writeback: convert wbc_account_cgroup_owner to take a folio

Most of the callers of wbc_account_cgroup_owner() are converting a folio
to page before calling the function. wbc_account_cgroup_owner(

fs/writeback: convert wbc_account_cgroup_owner to take a folio

Most of the callers of wbc_account_cgroup_owner() are converting a folio
to page before calling the function. wbc_account_cgroup_owner() is
converting the page back to a folio to call mem_cgroup_css_from_folio().

Convert wbc_account_cgroup_owner() to take a folio instead of a page,
and convert all callers to pass a folio directly except f2fs.

Convert the page to folio for all the callers from f2fs as they were the
only callers calling wbc_account_cgroup_owner() with a page. As f2fs is
already in the process of converting to folios, these call sites might
also soon be calling wbc_account_cgroup_owner() with a folio directly in
the future.

No functional changes. Only compile tested.

Signed-off-by: Pankaj Raghav <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Acked-by: David Sterba <[email protected]>
Acked-by: Tejun Heo <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

reiserfs: The last commit

Deprecation period of reiserfs ends with the end of this year so it is
time to remove it from the kernel.

Acked-by: Darrick J. Wong <[email protected]>
Acked-by: Christian

reiserfs: The last commit

Deprecation period of reiserfs ends with the end of this year so it is
time to remove it from the kernel.

Acked-by: Darrick J. Wong <[email protected]>
Acked-by: Christian Brauner <[email protected]>
Signed-off-by: Jan Kara <[email protected]>

show more ...

fs: Move clearing of mappedtodisk to buffer.c

The mappedtodisk flag is only meaningful for buffer head based
filesystems. It should not be cleared for other filesystems. This allows
us to reuse th

fs: Move clearing of mappedtodisk to buffer.c

The mappedtodisk flag is only meaningful for buffer head based
filesystems. It should not be cleared for other filesystems. This allows
us to reuse the mappedtodisk flag to have other meanings in filesystems
that do not use buffer heads.

Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: drop GFP_NOFAIL mode from alloc_page_buffers

There is only one called of alloc_page_buffers and it doesn't require
__GFP_NOFAIL so drop this allocation mode.

Signed-off-by: Michal Hocko <mhocko

fs: drop GFP_NOFAIL mode from alloc_page_buffers

There is only one called of alloc_page_buffers and it doesn't require
__GFP_NOFAIL so drop this allocation mode.

Signed-off-by: Michal Hocko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Acked-by: Song Liu <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: use LIST_HEAD() to simplify code

list_head can be initialized automatically with LIST_HEAD()
instead of calling INIT_LIST_HEAD().

Signed-off-by: Hongbo Li <[email protected]>
Link: https://

fs: use LIST_HEAD() to simplify code

list_head can be initialized automatically with LIST_HEAD()
instead of calling INIT_LIST_HEAD().

Signed-off-by: Hongbo Li <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

buffer: Convert __block_write_begin() to take a folio

Almost all callers have a folio now, so change __block_write_begin()
to take a folio and remove a call to compound_head().

Reviewed-by: Josef B

buffer: Convert __block_write_begin() to take a folio

Almost all callers have a folio now, so change __block_write_begin()
to take a folio and remove a call to compound_head().

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: Convert aops->write_begin to take a folio

Convert all callers from working on a page to working on one page
of a folio (support for working on an entire folio can come later).
Removes a lot of f

fs: Convert aops->write_begin to take a folio

Convert all callers from working on a page to working on one page
of a folio (support for working on an entire folio can come later).
Removes a lot of folio->page->folio conversions.

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: Convert aops->write_end to take a folio

Most callers have a folio, and most implementations operate on a folio,
so remove the conversion from folio->page->folio to fit through this
interface.

R

fs: Convert aops->write_end to take a folio

Most callers have a folio, and most implementations operate on a folio,
so remove the conversion from folio->page->folio to fit through this
interface.

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

buffer: Convert block_write_end() to take a folio

All callers now have a folio, so pass it in instead of converting
from a folio to a page and back to a folio again. Saves a call
to compound_head()

buffer: Convert block_write_end() to take a folio

All callers now have a folio, so pass it in instead of converting
from a folio to a page and back to a folio again. Saves a call
to compound_head().

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

buffer: Use a folio in generic_write_end()

Replaces two implicit calls to compound_head() with one.

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <willy@inf

buffer: Use a folio in generic_write_end()

Replaces two implicit calls to compound_head() with one.

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: Convert block_write_begin() to use a folio

Use the folio APIs to retrieve the folio from the page cache and
manipulate it. Saves a few conversions between pages & folios.

Reviewed-by: Josef Ba

fs: Convert block_write_begin() to use a folio

Use the folio APIs to retrieve the folio from the page cache and
manipulate it. Saves a few conversions between pages & folios.

Reviewed-by: Josef Bacik <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...