bcachefs: Use sort_nonatomic() instead of sort()

Fixes "task out to lunch" warnings during recovery on large machines
with lots of dirty data in the journal.

Signed-off-by: Kent Overstreet <kent.ov

bcachefs: Use sort_nonatomic() instead of sort()

Fixes "task out to lunch" warnings during recovery on large machines
with lots of dirty data in the journal.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: use library APIs for ChaCha20 and Poly1305

Just use the ChaCha20 and Poly1305 libraries instead of the clunky
crypto API. This is much simpler. It is also slightly faster, since
the libr

bcachefs: use library APIs for ChaCha20 and Poly1305

Just use the ChaCha20 and Poly1305 libraries instead of the clunky
crypto API. This is much simpler. It is also slightly faster, since
the libraries provide more direct access to the same
architecture-optimized ChaCha20 and Poly1305 code.

I've tested that existing encrypted bcachefs filesystems can be continue
to be accessed with this patch applied.

Reviewed-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Eric Biggers <[email protected]>
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Split up bch_dev.io_ref

We now have separate per device io_refs for read and write access.

This fixes a device removal bug where the discard workers were still
running while we're removin

bcachefs: Split up bch_dev.io_ref

We now have separate per device io_refs for read and write access.

This fixes a device removal bug where the discard workers were still
running while we're removing alloc info for that device.

It's also a bit of hardening; we no longer allow writes to devices that
are read-only.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Better printing of inconsistency errors

Build up and emit the error message for an inconsistency error all at
once, instead of spread over multiple printk calls, so they're not
jumbled in

bcachefs: Better printing of inconsistency errors

Build up and emit the error message for an inconsistency error all at
once, instead of spread over multiple printk calls, so they're not
jumbled in the dmesg log.

Also, add better indenting.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Fix 'hung task' messages in btree node scan

btree node scan has to wait on kthread workers that scan each device,
potentially for awhile.

We would like this to be interruptible, but we ma

bcachefs: Fix 'hung task' messages in btree node scan

btree node scan has to wait on kthread workers that scan each device,
potentially for awhile.

We would like this to be interruptible, but we may need a different
mechanism than signals for that - we've had bugs in the past where
mounts were failing due to checking for signals, and no explanation on
where they came from.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Fix btree_node_scan io_ref handling

This was completely fubar; it's now simplified a bit as well.
Note that for_each_online_member() takes and releases io_refs as it
iterates, so we need t

bcachefs: Fix btree_node_scan io_ref handling

This was completely fubar; it's now simplified a bit as well.
Note that for_each_online_member() takes and releases io_refs as it
iterates, so we need to release that if we break.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Finish bch2_account_io_completion() conversions

More prep work for automatically kicking devices out after too many IO
errors.

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Use a heap for handling overwrites in btree node scan

Fix an O(n^2) issue when we find many overlapping (overwritten) btree
nodes - especially when one node overwrites many smaller nodes.

bcachefs: Use a heap for handling overwrites in btree node scan

Fix an O(n^2) issue when we find many overlapping (overwritten) btree
nodes - especially when one node overwrites many smaller nodes.

This was discovered to be an issue with the bcachefs
merge_torture_flakey test - if we had a large btree that was then
emptied, the number of difficult overwrites can be unbounded.

Cc: Kuan-Wei Chiu <[email protected]>
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it

bcachefs: struct bkey_validate_context

Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it came from the journal, or is a btree root.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Don't try to en/decrypt when encryption not available

If a btree node says it's encrypted, but the superblock never had an
encryptino key - whoops, that needs to be handled.

Reported-by:

bcachefs: Don't try to en/decrypt when encryption not available

If a btree node says it's encrypted, but the superblock never had an
encryptino key - whoops, that needs to be handled.

Reported-by: [email protected]
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Avoid bch2_btree_id_str()

Prefer bch2_btree_id_to_text() - it prints out the integer ID when
unknown.

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Fix null ptr deref in bucket_gen_get()

bucket_gen() checks if we're lookup up a valid bucket and returns NULL
otherwise, but bucket_gen_get() was failing to check; other callers were
corre

bcachefs: Fix null ptr deref in bucket_gen_get()

bucket_gen() checks if we're lookup up a valid bucket and returns NULL
otherwise, but bucket_gen_get() was failing to check; other callers were
correct.

Also do a bit of cleanup on callers.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: add check for btree id against max in try read node

Add check for read node's btree_id against BTREE_ID_NR_MAX in
try_read_btree_node to prevent triggering EBUG_ON condition in
bch2_btree_

bcachefs: add check for btree id against max in try read node

Add check for read node's btree_id against BTREE_ID_NR_MAX in
try_read_btree_node to prevent triggering EBUG_ON condition in
bch2_btree_id_root[1].

[1] https://syzkaller.appspot.com/bug?extid=cf7b2215b5d70600ec00

Reported-by: [email protected]
Closes: https://syzkaller.appspot.com/bug?extid=cf7b2215b5d70600ec00
Fixes: 4409b8081d16 ("bcachefs: Repair pass for scanning for btree nodes")
Signed-off-by: Piotr Zalewski <[email protected]>
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Fix incorrect IS_ERR_OR_NULL usage

Returning a positive integer instead of an error code causes error paths
to become very confused.

Closes: [email protected]

bcachefs: Fix incorrect IS_ERR_OR_NULL usage

Returning a positive integer instead of an error code causes error paths
to become very confused.

Closes: [email protected]
Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Make bkey_fsck_err() a wrapper around fsck_err()

bkey_fsck_err() was added as an interface that looks like fsck_err(),
but previously all it did was ensure that the appropriate error count

bcachefs: Make bkey_fsck_err() a wrapper around fsck_err()

bkey_fsck_err() was added as an interface that looks like fsck_err(),
but previously all it did was ensure that the appropriate error counter
was incremented in the superblock.

This is a cleanup and bugfix patch that converts it to a wrapper around
fsck_err(). This is needed to fix an issue with the upgrade path to
disk_accounting_v3, where the "silent fix" error list now includes
bkey_fsck errors; fsck_err() handles this in a unified way, and since we
need to change printing of bkey fsck errors from the caller to the inner
bkey_fsck_err() calls, this ends up being a pretty big change.

Als,, rename .invalid() methods to .validate(), for clarity, while we're
changing the function signature anyways (to drop the printbuf argument).

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: btree node scan: fall back to comparing by journal seq

highly damaged filesystems, or filesystems that have been damaged and
repair and damaged again, may have sequence numbers we can't fu

bcachefs: btree node scan: fall back to comparing by journal seq

highly damaged filesystems, or filesystems that have been damaged and
repair and damaged again, may have sequence numbers we can't fully trust
- which in itself is something we need to debug.

Add a journal_seq fallback so that repair doesn't get stuck.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: Fix incorrect error handling found_btree_node_is_readable()

error handling here is slightly odd, which is why we were accidently
calling evict() on an error pointer

Signed-off-by: Kent Ov

bcachefs: Fix incorrect error handling found_btree_node_is_readable()

error handling here is slightly odd, which is why we were accidently
calling evict() on an error pointer

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: btree node scan now fills in sectors_written

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: node scan: ignore multiple nodes with same seq if interior

Interior nodes are not really needed, when we have to scan - but if this
pops up for leaf nodes we'll need a real heuristic.

Sig

bcachefs: node scan: ignore multiple nodes with same seq if interior

Interior nodes are not really needed, when we have to scan - but if this
pops up for leaf nodes we'll need a real heuristic.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: bch_member.btree_allocated_bitmap

This adds a small (64 bit) per-device bitmap that tracks ranges that
have btree nodes, for accelerating btree node scan if it is ever needed.

- New helpe

bcachefs: bch_member.btree_allocated_bitmap

This adds a small (64 bit) per-device bitmap that tracks ranges that
have btree nodes, for accelerating btree node scan if it is ever needed.

- New helpers, bch2_dev_btree_bitmap_marked() and
bch2_dev_bitmap_mark(), for checking and updating the bitmap

- Interior btree update path updates the bitmaps when required

- The check_allocations pass has a new fsck_err check,
btree_bitmap_not_marked

- New on disk format version, mi_btree_mitmap, which indicates the new
bitmap is present

- Upgrade table lists the required recovery pass and expected fsck error

- Btree node scan uses the bitmap to skip ranges if we're on the new
version

Signed-off-by: Kent Overstreet <[email protected]>

show more ...

bcachefs: btree node scan: handle encrypted nodes

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: btree_node_scan: Respect member.data_allowed

If a device wasn't used for btree nodes, no need to scan for them.

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Don't scan for btree nodes when we can reconstruct

Signed-off-by: Kent Overstreet <[email protected]>

bcachefs: Repair pass for scanning for btree nodes

If a btree root or interior btree node goes bad, we're going to lose a
lot of data, unless we can recover the nodes that it pointed to by
scanning.

bcachefs: Repair pass for scanning for btree nodes

If a btree root or interior btree node goes bad, we're going to lose a
lot of data, unless we can recover the nodes that it pointed to by
scanning.

Fortunately btree node headers are fully self describing, and
additionally the magic number is xored with the filesytem UUID, so we
can do so safely.

This implements the scanning - next patch will rework topology repair to
make use of the found nodes.

Signed-off-by: Kent Overstreet <[email protected]>

show more ...