fs/buffer fs/mpage: remove large folio restriction

Now that buffer-heads has been converted over to support large folios
we can remove the built-in VM_BUG_ON_FOLIO() checks which prevents
their use.

fs/buffer fs/mpage: remove large folio restriction

Now that buffer-heads has been converted over to support large folios
we can remove the built-in VM_BUG_ON_FOLIO() checks which prevents
their use.

Reviewed-by: Hannes Reinecke <[email protected]>
Reviewed-by: "Matthew Wilcox (Oracle)" <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: John Garry <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/mpage: use blocks_per_folio instead of blocks_per_page

Convert mpage to folios and adjust accounting for the number of blocks
within a folio instead of a single page. This also adjusts the number

fs/mpage: use blocks_per_folio instead of blocks_per_page

Convert mpage to folios and adjust accounting for the number of blocks
within a folio instead of a single page. This also adjusts the number
of pages we should process to be the size of the folio to ensure we
always read a full folio.

Note that the page cache code already ensures do_mpage_readpage() will
work with folios respecting the address space min order, this ensures
that so long as folio_size() is used for our requirements mpage will
also now be able to process block sizes larger than the page size.

Originally-by: Hannes Reinecke <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Hannes Reinecke <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/mpage: avoid negative shift for large blocksize

For large blocksizes the number of block bits is larger than PAGE_SHIFT,
so calculate the sector number from the byte offset instead. This is
requi

fs/mpage: avoid negative shift for large blocksize

For large blocksizes the number of block bits is larger than PAGE_SHIFT,
so calculate the sector number from the byte offset instead. This is
required to enable large folios with buffer-heads.

Reviewed-by: "Matthew Wilcox (Oracle)" <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
Signed-off-by: Hannes Reinecke <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Hannes Reinecke <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs/writeback: convert wbc_account_cgroup_owner to take a folio

Most of the callers of wbc_account_cgroup_owner() are converting a folio
to page before calling the function. wbc_account_cgroup_owner(

fs/writeback: convert wbc_account_cgroup_owner to take a folio

Most of the callers of wbc_account_cgroup_owner() are converting a folio
to page before calling the function. wbc_account_cgroup_owner() is
converting the page back to a folio to call mem_cgroup_css_from_folio().

Convert wbc_account_cgroup_owner() to take a folio instead of a page,
and convert all callers to pass a folio directly except f2fs.

Convert the page to folio for all the callers from f2fs as they were the
only callers calling wbc_account_cgroup_owner() with a page. As f2fs is
already in the process of converting to folios, these call sites might
also soon be calling wbc_account_cgroup_owner() with a folio directly in
the future.

No functional changes. Only compile tested.

Signed-off-by: Pankaj Raghav <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Acked-by: David Sterba <[email protected]>
Acked-by: Tejun Heo <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>

show more ...

buffer: Remove calls to set and clear the folio error flag

The folio error flag is not tested anywhere, so we can stop setting
and clearing it.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infrade

buffer: Remove calls to set and clear the folio error flag

The folio error flag is not tested anywhere, so we can stop setting
and clearing it.

Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Christian Brauner <[email protected]>

show more ...

block, fs: Restore the per-bio/request data lifetime fields

Restore support for passing data lifetime information from filesystems to
block drivers. This patch reverts commit b179c98f7697 ("block: R

block, fs: Restore the per-bio/request data lifetime fields

Restore support for passing data lifetime information from filesystems to
block drivers. This patch reverts commit b179c98f7697 ("block: Remove
request.write_hint") and commit c75e707fe1aa ("block: remove the
per-bio/request write hint").

This patch does not modify the size of struct bio because the new
bi_write_hint member fills a hole in struct bio. pahole reports the
following for struct bio on an x86_64 system with this patch applied:

/* size: 112, cachelines: 2, members: 20 */
/* sum members: 110, holes: 1, sum holes: 2 */
/* last cacheline: 48 bytes */

Reviewed-by: Kanchan Joshi <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Signed-off-by: Bart Van Assche <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Christian Brauner <[email protected]>

show more ...

fs: convert block_write_full_page to block_write_full_folio

Convert the function to be compatible with writepage_t so that it can be
passed to write_cache_pages() by blkdev. This removes a call to

fs: convert block_write_full_page to block_write_full_folio

Convert the function to be compatible with writepage_t so that it can be
passed to write_cache_pages() by blkdev. This removes a call to
compound_head(). We can also remove the function export as both callers
are built-in.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jens Axboe <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: reduce stack usage in do_mpage_readpage

Some architectures support a very large PAGE_SIZE, so instead of the 8
pointers we see with a 4kB PAGE_SIZE, we can see 128 pointers with 64kB or
so many

fs: reduce stack usage in do_mpage_readpage

Some architectures support a very large PAGE_SIZE, so instead of the 8
pointers we see with a 4kB PAGE_SIZE, we can see 128 pointers with 64kB or
so many on Hexagon that it trips compiler warnings about exceeding stack
frame size.

All we're doing with this array is checking for block contiguity, which we
can as well do by remembering the address of the first block in the page
and checking this block is at the appropriate offset from that address.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jens Axboe <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: reduce stack usage in __mpage_writepage

Some architectures support a very large PAGE_SIZE, so instead of the 8
pointers we see with a 4kB PAGE_SIZE, we can see 128 pointers with 64kB or
so many

fs: reduce stack usage in __mpage_writepage

Some architectures support a very large PAGE_SIZE, so instead of the 8
pointers we see with a 4kB PAGE_SIZE, we can see 128 pointers with 64kB or
so many on Hexagon that it trips compiler warnings about exceeding stack
frame size.

All we're doing with this array is checking for block contiguity, which we
can as well do by remembering the address of the first block in the page
and checking this block is at the appropriate offset from that address.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jens Axboe <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: convert clean_buffers() to take a folio

The only caller already has a folio, so pass it in and use it throughout.
Saves two calls to compound_head().

Link: https://lkml.kernel.org/r/2023121520

fs: convert clean_buffers() to take a folio

The only caller already has a folio, so pass it in and use it throughout.
Saves two calls to compound_head().

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jens Axboe <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: remove clean_page_buffers()

Patch series "Clean up the writeback paths".

Most of these patches verge on the trivial, converting filesystems that
just use block_write_full_page() to use mpage_wr

fs: remove clean_page_buffers()

Patch series "Clean up the writeback paths".

Most of these patches verge on the trivial, converting filesystems that
just use block_write_full_page() to use mpage_writepages(). But as we saw
with Christoph's earlier patchset, there can be some "interesting"
gotchas, and I clearly haven't tested the majority of filesystems I've
touched here.

Patches 3 & 4 get rid of a lot of stack usage on architectures with larger
page sizes; 1024 bytes on 64-bit systems with 64KiB pages. It starts to
open the door to larger folio sizes on all architectures, but it's
certainly not enough yet.

Patch 14 is kind of trivial, but it's nice to get that simplification in.


This patch (of 14):

This function has been unused since the removal of bdev_write_page().

Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jens Axboe <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

buffer: remove folio_create_empty_buffers()

With all users converted, remove the old create_empty_buffers() and rename
folio_create_empty_buffers() to create_empty_buffers().

Link: https://lkml.ker

buffer: remove folio_create_empty_buffers()

With all users converted, remove the old create_empty_buffers() and rename
folio_create_empty_buffers() to create_empty_buffers().

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Cc: Andreas Gruenbacher <[email protected]>
Cc: Pankaj Raghav <[email protected]>
Cc: Ryusuke Konishi <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: convert map_buffer_to_folio() to folio_create_empty_buffers()

Saves a folio->page->folio conversion.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-b

mpage: convert map_buffer_to_folio() to folio_create_empty_buffers()

Saves a folio->page->folio conversion.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Pankaj Raghav <[email protected]>
Cc: Andreas Gruenbacher <[email protected]>
Cc: Ryusuke Konishi <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: use folios in bio end_io handler

Use folios in the bio end_io handler. This conversion does the
appropriate handling on the folios in the respective end_io callback and
removes the call to p

mpage: use folios in bio end_io handler

Use folios in the bio end_io handler. This conversion does the
appropriate handling on the folios in the respective end_io callback and
removes the call to page_endio(), which is soon to be removed.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Pankaj Raghav <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Cc: Alexander Viro <[email protected]>
Cc: Christian Brauner <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Luis Chamberlain <[email protected]>
Cc: Martin Brandenburg <[email protected]>
Cc: Matthew Wilcox (Oracle) <[email protected]>
Cc: Mike Marshall <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: split submit_bio and bio end_io handler for reads and writes

Split the submit_bio() and bio end_io handler for reads and writes similar
to other aops.

This is a prep patch before we convert

mpage: split submit_bio and bio end_io handler for reads and writes

Split the submit_bio() and bio end_io handler for reads and writes similar
to other aops.

This is a prep patch before we convert end_io handlers to use folios.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Pankaj Raghav <[email protected]>
Suggested-by: Christoph Hellwig <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Cc: Alexander Viro <[email protected]>
Cc: Christian Brauner <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Luis Chamberlain <[email protected]>
Cc: Martin Brandenburg <[email protected]>
Cc: Matthew Wilcox (Oracle) <[email protected]>
Cc: Mike Marshall <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: gracefully handle ->get_block not mapping bh in __mpage_writepage

When filesystem's ->get_block function does not map the buffer head when
called from __mpage_writepage(), __mpage_writepage() wi

fs: gracefully handle ->get_block not mapping bh in __mpage_writepage

When filesystem's ->get_block function does not map the buffer head when
called from __mpage_writepage(), __mpage_writepage() will happily go and
pass bogus bdev and block number to bio allocation routines which leads to
crashes sooner or later.

E.g. UDF can do this because it doesn't want to allocate blocks from
->writepages callbacks. It allocates blocks on write or page fault but
writeback can still spot dirty buffers without underlying blocks allocated
e.g. if blocksize < pagesize, the tail page is dirtied (which means all
its buffers are dirtied), and truncate extends the file so that some
buffer starts to be within i_size.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Jan Kara <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: convert __mpage_writepage() to use a folio more fully

This is just a conversion to the folio API. While there are some nods
towards supporting multi-page folios in here, the blocks array is

mpage: convert __mpage_writepage() to use a folio more fully

This is just a conversion to the folio API. While there are some nods
towards supporting multi-page folios in here, the blocks array is still
sized for one page's worth of blocks, and there are other assumptions such
as the blocks_per_page variable.

[[email protected]: fix accidentally-triggering WARN_ON_ONCE]
Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Jan Kara <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: convert writepage_t callback to pass a folio

Patch series "Convert writepage_t to use a folio".

More folioisation. I split out the mpage work from everything else
because it completely dominat

fs: convert writepage_t callback to pass a folio

Patch series "Convert writepage_t to use a folio".

More folioisation. I split out the mpage work from everything else
because it completely dominated the patch, but some implementations I just
converted outright.


This patch (of 2):

We always write back an entire folio, but that's currently passed as the
head page. Convert all filesystems that use write_cache_pages() to expect
a folio instead of a page.

Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: stop using bdev_{read,write}_page

Patch series "remove ->rw_page".

This series removes the ->rw_page block_device_operation, which is an old
and clumsy attempt at a simple read/write fast pa

mpage: stop using bdev_{read,write}_page

Patch series "remove ->rw_page".

This series removes the ->rw_page block_device_operation, which is an old
and clumsy attempt at a simple read/write fast path for the block layer.
It isn't actually used by the fastest block layer operations that we
support (polled I/O through io_uring), but only used by the mpage buffered
I/O helpers which are some of the slowest I/O we have and do not make any
difference there at all, and zram which is a block device abused to
duplicate the zram functionality.

Given that zram is heavily used we need to make sure there is a good
replacement for synchronous I/O, so this series adds a new flag for
drivers that complete I/O synchronously and uses that flag to use on-stack
bios and synchronous submission for them in the swap code.


This patch (of 7):

These are micro-optimizations for synchronous I/O, which do not matter
compared to all the other inefficiencies in the legacy buffer_head based
mpage code.

Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Dan Williams <[email protected]>
Cc: Keith Busch <[email protected]>
Cc: Dave Jiang <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
Cc: Vishal Verma <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: gracefully handle ->get_block not mapping bh in __mpage_writepage

When filesystem's ->get_block function does not map the buffer head when
called from __mpage_writepage(), the function will happ

fs: gracefully handle ->get_block not mapping bh in __mpage_writepage

When filesystem's ->get_block function does not map the buffer head when
called from __mpage_writepage(), the function will happily go and pass
bogus bdev and block number to bio allocation routines which leads to
crashes sooner or later. E.g. UDF can do this because it doesn't want to
allocate blocks from ->writepages callbacks. It allocates blocks on
write or page fault but writeback can still spot dirty buffers without
underlying blocks allocated e.g. if blocksize < pagesize, the tail page
is dirtied (which means all its buffers are dirtied), and truncate
extends the file so that some buffer starts to be within i_size.

Reviewed-by: Christoph Hellwig <[email protected]>
Signed-off-by: Jan Kara <[email protected]>

show more ...

fs: don't allocate blocks beyond EOF from __mpage_writepage

When __mpage_writepage() is called for a page beyond EOF, it will go and
allocate all blocks underlying the page. This is not only unnece

fs: don't allocate blocks beyond EOF from __mpage_writepage

When __mpage_writepage() is called for a page beyond EOF, it will go and
allocate all blocks underlying the page. This is not only unnecessary but
this way blocks can get leaked (e.g. if a page beyond EOF is marked dirty
but in the end write fails and i_size is not extended).

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Jan Kara <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Al Viro <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: remove an outdated comment on mpage_writepages

Patch series "remove generic_writepages"

This series removes generic_writepages by open coding the current
functionality in the three remaining ca

fs: remove an outdated comment on mpage_writepages

Patch series "remove generic_writepages"

This series removes generic_writepages by open coding the current
functionality in the three remaining callers. Besides removing some
code the main benefit is that one of the few remaining ->writepage
callers from outside the core page cache code go away.

This patch (of 6):


mpage_writepages doesn't do any of the page locking itself, so remove and
outdated comment on the locking pattern there.

Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Cc: Joel Becker <[email protected]>
Cc: Joseph Qi <[email protected]>
Cc: Konstantin Komarov <[email protected]>
Cc: Mark Fasheh <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

mpage: use b_folio in do_mpage_readpage()

Remove this conversion of a folio back to a page.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilc

mpage: use b_folio in do_mpage_readpage()

Remove this conversion of a folio back to a page.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

fs: remove the NULL get_block case in mpage_writepages

No one calls mpage_writepages with a NULL get_block paramter, so remove
support for that case.

Signed-off-by: Christoph Hellwig <[email protected]>
R

fs: remove the NULL get_block case in mpage_writepages

No one calls mpage_writepages with a NULL get_block paramter, so remove
support for that case.

Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>

show more ...

fs: don't call ->writepage from __mpage_writepage

All callers of mpage_writepage use block_write_full_page as their
->writepage implementation when called from mpage_writepages
(although for ntfs3 t

fs: don't call ->writepage from __mpage_writepage

All callers of mpage_writepage use block_write_full_page as their
->writepage implementation when called from mpage_writepages
(although for ntfs3 this is obsfucated a bit).

Just call block_write_full_page directly instead of going through
the ->writepage indirection.

Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>

show more ...