bcachefs: bcachefs_metadata_version_stripe_backpointers

Stripes now have backpointers.

This is needed for proper scrub - stripe checksums need to be verified,
separately from extents within the str

bcachefs: bcachefs_metadata_version_stripe_backpointers

Stripes now have backpointers.

This is needed for proper scrub - stripe checksums need to be verified,
separately from extents within the stripe, since a block may not be full
of live extents but it's still needed for reconstruct.

And this will be needed for (efficient) evacuate/repair paths.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bch2_bkey_ptr_data_type() now correctly returns cached for cached ptrs

Necessary for adding backpointers for cached pointers.

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Factor out progress.[ch]

the backpointers code has progress indicators; these aren't great, since
they print to the dmesg console and we much prefer to have progress
indicators reporting t

bcachefs: Factor out progress.[ch]

the backpointers code has progress indicators; these aren't great, since
they print to the dmesg console and we much prefer to have progress
indicators reporting to a specific userspace program so they're not
spamming the system console.

But not all codepaths that need progress indicators support that yet,
and we don't want users to think "this is hung".

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Add write buffer flush param to backpointer_get_key()

In an upcoming patch bch2_backpointer_get_key() will be repairing when
it finds a dangling backpointer; it will need to flush the btre

bcachefs: Add write buffer flush param to backpointer_get_key()

In an upcoming patch bch2_backpointer_get_key() will be repairing when
it finds a dangling backpointer; it will need to flush the btree write
buffer before it can definitively say there's an error.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: kill __bch2_extent_ptr_to_bp()

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: bch2_extent_ptr_to_bp() no longer depends on device

bch_backpointer no longer contains the bucket_offset field, it's just a
direct LBA mapping (with low bits to account for compressed exte

bcachefs: bch2_extent_ptr_to_bp() no longer depends on device

bch_backpointer no longer contains the bucket_offset field, it's just a
direct LBA mapping (with low bits to account for compressed extent
splitting), so we don't need to refer to the device to construct it
anymore.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bcachefs_metadata_version_backpointer_bucket_gen

New on disk format version: backpointers new include the generation
number of the bucket they refer to, and the obsolete bucket_offset fiel

bcachefs: bcachefs_metadata_version_backpointer_bucket_gen

New on disk format version: backpointers new include the generation
number of the bucket they refer to, and the obsolete bucket_offset field
(no longer needed because we no longer store backpointers in alloc keys)
is gone.

This is an expensive forced upgrade - hopefully the last; we have to run
the extents_to_backpointers recovery pass to regenerate backpointers.

It's a forced incompatible upgrade because the alternative would've been
permamently making backpointers bigger, and as one of the biggest btrees
(along with the extents btree) that's not an ideal option.

It's worth it though, because this allows us to make the
check_extents_to_backpointers pass drastically cheaper: an upcoming
patch changes it to sum up backpointers in a bucket and check the sum
against the sector counts for that bucket, only looking for missing
backpointers if they don't match (and then only for specific buckets).

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it came from the journal, or is a btree root.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: kill bch_backpointer.bucket_offset usage

bch_backpointer.bucket_offset is going away - it's no longer needed
since we no longer store backpointers in alloc keys, the same
information is in

bcachefs: kill bch_backpointer.bucket_offset usage

bch_backpointer.bucket_offset is going away - it's no longer needed
since we no longer store backpointers in alloc keys, the same
information is in the key position itself.

And we'll be reclaiming the space in bch_backpointer for the bucket
generation number.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bch_backpointer -> bkey_i_backpointer

Since we no longer store backpointers in alloc keys, there's no reason
not to pass around bkey_i_backpointers; this means we don't have to pass
the bu

bcachefs: bch_backpointer -> bkey_i_backpointer

Since we no longer store backpointers in alloc keys, there's no reason
not to pass around bkey_i_backpointers; this means we don't have to pass
the bucket pos separately.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bucket_pos_to_bp_end()

Better helpers for iterating over backpointers within a specific bucket

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: fix bp_pos_to_bucket_nodev_noerror

_noerror means don't produce inconsistent errors, so it should be using
bch2_dev_rcu_noerror().

Signed-off-by: Kent Overstreet <[email protected]

bcachefs: fix bp_pos_to_bucket_nodev_noerror

_noerror means don't produce inconsistent errors, so it should be using
bch2_dev_rcu_noerror().

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Kill bch2_get_next_backpointer()

Since for quite some time backpointers have only been stored in the
backpointers btree, not alloc keys (an aborted experiment, support for
which has been r

bcachefs: Kill bch2_get_next_backpointer()

Since for quite some time backpointers have only been stored in the
backpointers btree, not alloc keys (an aborted experiment, support for
which has been removed) - we can replace get_next_backpointer() with
simple btree iteration.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bch2_trigger_ptr() calculates sectors even when no device

This is necessary for erasure coded pointers to devices that have been
removed.

Signed-off-by: Kent Overstreet <kent.overstreet@l

bcachefs: bch2_trigger_ptr() calculates sectors even when no device

This is necessary for erasure coded pointers to devices that have been
removed.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Make bkey_fsck_err() a wrapper around fsck_err()

bkey_fsck_err() was added as an interface that looks like fsck_err(),
but previously all it did was ensure that the appropriate error count

bcachefs: Make bkey_fsck_err() a wrapper around fsck_err()

bkey_fsck_err() was added as an interface that looks like fsck_err(),
but previously all it did was ensure that the appropriate error counter
was incremented in the superblock.

This is a cleanup and bugfix patch that converts it to a wrapper around
fsck_err(). This is needed to fix an issue with the upgrade path to
disk_accounting_v3, where the "silent fix" error list now includes
bkey_fsck errors; fsck_err() handles this in a unified way, and since we
need to change printing of bkey fsck errors from the caller to the inner
bkey_fsck_err() calls, this ends up being a pretty big change.

Als,, rename .invalid() methods to .validate(), for clarity, while we're
changing the function signature anyways (to drop the printbuf argument).

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: s/bkey_invalid_flags/bch_validate_flags

We're about to start using bch_validate_flags for superblock section
validation - it's no longer bkey specific.

Signed-off-by: Kent Overstreet <ken

bcachefs: s/bkey_invalid_flags/bch_validate_flags

We're about to start using bch_validate_flags for superblock section
validation - it's no longer bkey specific.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Kill bch2_dev_bkey_exists() in backpointer code

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: PTR_BUCKET_POS() now takes bch_dev

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: member helper cleanups

Some renaming for better consistency

bch2_member_exists -> bch2_member_alive
bch2_dev_exists -> bch2_member_exists
bch2_dev_exsits2 -> bch2_dev_exists
bch_dev_lock

bcachefs: member helper cleanups

Some renaming for better consistency

bch2_member_exists -> bch2_member_alive
bch2_dev_exists -> bch2_member_exists
bch2_dev_exsits2 -> bch2_dev_exists
bch_dev_locked -> bch2_dev_locked
bch_dev_bkey_exists -> bch2_dev_bkey_exists

new helper - bch2_dev_safe

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bucket_pos_to_bp_noerror()

We don't want the assert when we're checking if the backpointer is
valid.

Reported-by: [email protected]
Signed-off-by: Kent

bcachefs: bucket_pos_to_bp_noerror()

We don't want the assert when we're checking if the backpointer is
valid.

Reported-by: [email protected]
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Check for backpointer bucket_offset >= bucket size

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Kill bch2_bkey_ptr_data_type()

Remove some duplication, and inconsistency between check_fix_ptrs and
the main ptr marking paths

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Prep work for variable size btree node buffers

bcachefs btree nodes are big - typically 256k - and btree roots are
pinned in memory. As we're now up to 18 btrees, we now have significant
m

bcachefs: Prep work for variable size btree node buffers

bcachefs btree nodes are big - typically 256k - and btree roots are
pinned in memory. As we're now up to 18 btrees, we now have significant
memory overhead in mostly empty btree roots.

And in the future we're going to start enforcing that certain btree node
boundaries exist, to solve lock contention issues - analagous to XFS's
AGIs.

Thus, we need to start allocating smaller btree node buffers when we
can. This patch changes code that refers to the filesystem constant
c->opts.btree_node_size to refer to the btree node buffer size -
btree_buf_bytes() - where appropriate.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: No need to allocate keys for write buffer

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: use swab40 for bch_backpointer.bucket_offset bitfield

The bucket_offset field of bch_backpointer is a 40-bit bitfield, but the
bch2_backpointer_swab() helper uses swab32. This leads to inc

bcachefs: use swab40 for bch_backpointer.bucket_offset bitfield

The bucket_offset field of bch_backpointer is a 40-bit bitfield, but the
bch2_backpointer_swab() helper uses swab32. This leads to inconsistency
when an on-disk fs is accessed from an opposite endian machine.

As it turns out, we already have an internal swab40() helper that is
used from the bch_alloc_v4 swab callback. Lift it into the backpointers
header file and use it consistently in both places.

Signed-off-by: Brian Foster <[email protected]>
Signed-off-by: Kent Overstreet <[email protected]>

show more ...