bcachefs: Fix missing commit in backpointer to missing target

Fsck wants to do transaction commits from an outer context; it may have
other repair to do (i.e. duplicate backpointers).

But when call

bcachefs: Fix missing commit in backpointer to missing target

Fsck wants to do transaction commits from an outer context; it may have
other repair to do (i.e. duplicate backpointers).

But when calling backpointer_not_found() from runtime code, i.e. runtime
self healing, we should be doing the commit - the outer context expects
to just be doing lookups.

This fixes bugs where we get stuck spinning, reported as "RCU lock hold
time warnings.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: backpointer_get_key: check for null from peek_slot()

peek_slot() doesn't normally return bkey_s_c_null - except when we ask
for a key at a btree level that doesn't exist, which can happen

bcachefs: backpointer_get_key: check for null from peek_slot()

peek_slot() doesn't normally return bkey_s_c_null - except when we ask
for a key at a btree level that doesn't exist, which can happen here.

We might want to revisit this, but we'll have to look over all the
places where we use peek_slot() on interior nodes.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Kill btree_iter.trans

This was planned to be done ages ago, now finally completed; there are
places where we have quite a few btree_trans objects on the stack, so
this reduces stack usage

bcachefs: Kill btree_iter.trans

This was planned to be done ages ago, now finally completed; there are
places where we have quite a few btree_trans objects on the stack, so
this reduces stack usage somewhat.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Split up bch_dev.io_ref

We now have separate per device io_refs for read and write access.

This fixes a device removal bug where the discard workers were still
running while we're removin

bcachefs: Split up bch_dev.io_ref

We now have separate per device io_refs for read and write access.

This fixes a device removal bug where the discard workers were still
running while we're removing alloc info for that device.

It's also a bit of hardening; we no longer allow writes to devices that
are read-only.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Better printing of inconsistency errors

Build up and emit the error message for an inconsistency error all at
once, instead of spread over multiple printk calls, so they're not
jumbled in

bcachefs: Better printing of inconsistency errors

Build up and emit the error message for an inconsistency error all at
once, instead of spread over multiple printk calls, so they're not
jumbled in the dmesg log.

Also, add better indenting.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Consistent indentation of multiline fsck errors

Add the new helper printbuf_indent_add_nextline(), and use it in
__bch2_fsck_err() to centralize setting the indentation of multiline
fsck e

bcachefs: Consistent indentation of multiline fsck errors

Add the new helper printbuf_indent_add_nextline(), and use it in
__bch2_fsck_err() to centralize setting the indentation of multiline
fsck errors.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Recovery no longer holds state_lock

state_lock guards against devices coming or leaving, changing state, or
the filesystem changing between ro <-> rw.

But it's not necessary for running r

bcachefs: Recovery no longer holds state_lock

state_lock guards against devices coming or leaving, changing state, or
the filesystem changing between ro <-> rw.

But it's not necessary for running recovery passes, and holding it
blocks asynchronous events that would cause us to go RO or kick out
devices.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Handle backpointers with unknown data types

New data types might be added later, so we don't want to disallow
unknown data types - that'll be a compatibility hassle later. Instead,
ignore

bcachefs: Handle backpointers with unknown data types

New data types might be added later, so we don't want to disallow
unknown data types - that'll be a compatibility hassle later. Instead,
ignore them.

Reported-by: [email protected]
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Count BCH_DATA_parity backpointers correctly

These are counted as stripe data in the corresponding alloc keys.

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: bcachefs_metadata_version_cached_backpointers

Cached pointers now have backpointers.

This means that we'll be able to kill cached pointers in the
bucket_invalidate path, when invalidating

bcachefs: bcachefs_metadata_version_cached_backpointers

Cached pointers now have backpointers.

This means that we'll be able to kill cached pointers in the
bucket_invalidate path, when invalidating/reusing buckets containing
cached data, instead of leaving them around to be cleaned up by gc_gens
garbago collection - which requires a full metadata scan.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Factor out progress.[ch]

the backpointers code has progress indicators; these aren't great, since
they print to the dmesg console and we much prefer to have progress
indicators reporting t

bcachefs: Factor out progress.[ch]

the backpointers code has progress indicators; these aren't great, since
they print to the dmesg console and we much prefer to have progress
indicators reporting to a specific userspace program so they're not
spamming the system console.

But not all codepaths that need progress indicators support that yet,
and we don't want users to think "this is hung".

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: backpointer_get_key() doesn't pull in btree node

We may not need to pull in a btree node when walking backpointers -
don't do so unnecessarily when using backpointer_get_key().

It'll stil

bcachefs: backpointer_get_key() doesn't pull in btree node

We may not need to pull in a btree node when walking backpointers -
don't do so unnecessarily when using backpointer_get_key().

It'll still fall back to backpointer_get_node() in a few situations,
including btree roots (where an iterator can't point at just the key),
and races due to the interior update path not having deleted a
backpointer to an old node yet.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: check_bp_exists() check for backpointers for stale pointers

Early version of 'bcachefs_metadata_version_cached_backpointers' was
creating backpointers for stale cached pointers - whoops. N

bcachefs: check_bp_exists() check for backpointers for stale pointers

Early version of 'bcachefs_metadata_version_cached_backpointers' was
creating backpointers for stale cached pointers - whoops. Now we have to
repair those.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: better check_bp_exists() error message

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: better backpointer_target_not_found() error message

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: bch2_backpointer_get_key() now repairs dangling backpointers

Continuing on with the self healing theme, we should be running any
check and repair code at runtime that we can - instead of d

bcachefs: bch2_backpointer_get_key() now repairs dangling backpointers

Continuing on with the self healing theme, we should be running any
check and repair code at runtime that we can - instead of declaring the
filesystemt inconsistent.

This will also let us skip running the backpointers -> extents fsck pass
except in debug mode.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: check_extents_to_backpointers() now only checks buckets with mismatches

Instead of walking every extent and every backpointer it points to,
first sum up backpointers in each bucket and che

bcachefs: check_extents_to_backpointers() now only checks buckets with mismatches

Instead of walking every extent and every backpointer it points to,
first sum up backpointers in each bucket and check for mismatches, and
only look for missing backpointers if mismatches were detected, and only
check extents in those buckets.

This is a major fsck scalability improvement, since the two backpointers
passes (backpointers -> extents and extents -> backpointers) are the
most expensive fsck passes by far.

Additionally, to speed up the upgrade for backpointer bucket gens, or in
situations when we have to rebuild alloc info, add a special case for
when no backpointers are found in a bucket - don't check each individual
backpointer (in particular, avoiding the write buffer flushes), just
recreate them.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Add write buffer flush param to backpointer_get_key()

In an upcoming patch bch2_backpointer_get_key() will be repairing when
it finds a dangling backpointer; it will need to flush the btre

bcachefs: Add write buffer flush param to backpointer_get_key()

In an upcoming patch bch2_backpointer_get_key() will be repairing when
it finds a dangling backpointer; it will need to flush the btree write
buffer before it can definitively say there's an error.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bch2_extent_ptr_to_bp() no longer depends on device

bch_backpointer no longer contains the bucket_offset field, it's just a
direct LBA mapping (with low bits to account for compressed exte

bcachefs: bch2_extent_ptr_to_bp() no longer depends on device

bch_backpointer no longer contains the bucket_offset field, it's just a
direct LBA mapping (with low bits to account for compressed extent
splitting), so we don't need to refer to the device to construct it
anymore.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bcachefs_metadata_version_backpointer_bucket_gen

New on disk format version: backpointers new include the generation
number of the bucket they refer to, and the obsolete bucket_offset fiel

bcachefs: bcachefs_metadata_version_backpointer_bucket_gen

New on disk format version: backpointers new include the generation
number of the bucket they refer to, and the obsolete bucket_offset field
(no longer needed because we no longer store backpointers in alloc keys)
is gone.

This is an expensive forced upgrade - hopefully the last; we have to run
the extents_to_backpointers recovery pass to regenerate backpointers.

It's a forced incompatible upgrade because the alternative would've been
permamently making backpointers bigger, and as one of the biggest btrees
(along with the extents btree) that's not an ideal option.

It's worth it though, because this allows us to make the
check_extents_to_backpointers pass drastically cheaper: an upcoming
patch changes it to sum up backpointers in a bucket and check the sum
against the sector counts for that bucket, only looking for missing
backpointers if they don't match (and then only for specific buckets).

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: trace_write_buffer_maybe_flush

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Guard against backpointers to unknown btrees

Reported-by: [email protected]
Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it came from the journal, or is a btree root.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: New backpointers helpers

- bch2_backpointer_del()
- bch2_backpointer_maybe_flush()

Kill a bit of open coding and make sure we're properly handling the
btree write buffer.

Signed-off-by:

bcachefs: New backpointers helpers

- bch2_backpointer_del()
- bch2_backpointer_maybe_flush()

Kill a bit of open coding and make sure we're properly handling the
btree write buffer.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: kill bch_backpointer.bucket_offset usage

bch_backpointer.bucket_offset is going away - it's no longer needed
since we no longer store backpointers in alloc keys, the same
information is in

bcachefs: kill bch_backpointer.bucket_offset usage

bch_backpointer.bucket_offset is going away - it's no longer needed
since we no longer store backpointers in alloc keys, the same
information is in the key position itself.

And we'll be reclaiming the space in bch_backpointer for the bucket
generation number.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...