fuse: add kernel-enforced timeout option for requests

There are situations where fuse servers can become unresponsive or
stuck, for example if the server is deadlocked. Currently, there's no
good wa

fuse: add kernel-enforced timeout option for requests

There are situations where fuse servers can become unresponsive or
stuck, for example if the server is deadlocked. Currently, there's no
good way to detect if a server is stuck and needs to be killed manually.

This commit adds an option for enforcing a timeout (in seconds) for
requests where if the timeout elapses without the server responding to
the request, the connection will be automatically aborted.

Please note that these timeouts are not 100% precise. For example, the
request may take roughly an extra FUSE_TIMEOUT_TIMER_FREQ seconds beyond
the requested timeout due to internal implementation, in order to
mitigate overhead.

[SzM: Bump the API version number]

Signed-off-by: Joanne Koong <[email protected]>
Reviewed-by: Jeff Layton <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: removed unused function fuse_uring_create() from header

Function fuse_uring_create() is used only from dev_uring.c and does not
need to be exposed in the header file. Furthermore, it has the

fuse: removed unused function fuse_uring_create() from header

Function fuse_uring_create() is used only from dev_uring.c and does not
need to be exposed in the header file. Furthermore, it has the wrong
signature.

While there, also remove the 'struct fuse_ring' forward declaration.

Signed-off-by: Luis Henriques <[email protected]>
Reviewed-by: Bernd Schubert <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: {io-uring} Fix a possible req cancellation race

task-A (application) might be in request_wait_answer and
try to remove the request when it has FR_PENDING set.

task-B (a fuse-server io-uring t

fuse: {io-uring} Fix a possible req cancellation race

task-A (application) might be in request_wait_answer and
try to remove the request when it has FR_PENDING set.

task-B (a fuse-server io-uring task) might handle this
request with FUSE_IO_URING_CMD_COMMIT_AND_FETCH, when
fetching the next request and accessed the req from
the pending list in fuse_uring_ent_assign_req().
That code path was not protected by fiq->lock and so
might race with task-A.

For scaling reasons we better don't use fiq->lock, but
add a handler to remove canceled requests from the queue.

This also removes usage of fiq->lock from
fuse_uring_add_req_to_ring_ent() altogether, as it was
there just to protect against this race and incomplete.

Also added is a comment why FR_PENDING is not cleared.

Fixes: c090c8abae4b ("fuse: Add io-uring sqe commit and fetch support")
Cc: <[email protected]> # v6.14
Reported-by: Joanne Koong <[email protected]>
Closes: https://lore.kernel.org/all/CAJnrk1ZgHNb78dz-yfNTpxmW7wtT88A=m-zF0ZoLXKLUHRjNTw@mail.gmail.com/
Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Joanne Koong <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: {io-uring} Prevent mount point hang on fuse-server termination

When the fuse-server terminates while the fuse-client or kernel
still has queued URING_CMDs, these commands retain references
to

fuse: {io-uring} Prevent mount point hang on fuse-server termination

When the fuse-server terminates while the fuse-client or kernel
still has queued URING_CMDs, these commands retain references
to the struct file used by the fuse connection. This prevents
fuse_dev_release() from being invoked, resulting in a hung mount
point.

This patch addresses the issue by making queued URING_CMDs
cancelable, allowing fuse_dev_release() to proceed as expected
and preventing the mount point from hanging.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: Allow to queue bg requests through io-uring

This prepares queueing and sending background requests through
io-uring.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begun

fuse: Allow to queue bg requests through io-uring

This prepares queueing and sending background requests through
io-uring.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: Allow to queue fg requests through io-uring

This prepares queueing and sending foreground requests through
io-uring.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begun

fuse: Allow to queue fg requests through io-uring

This prepares queueing and sending foreground requests through
io-uring.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: {io-uring} Handle teardown of ring entries

On teardown struct file_operations::uring_cmd requests
need to be completed by calling io_uring_cmd_done().
Not completing all ring entries would res

fuse: {io-uring} Handle teardown of ring entries

On teardown struct file_operations::uring_cmd requests
need to be completed by calling io_uring_cmd_done().
Not completing all ring entries would result in busy io-uring
tasks giving warning messages in intervals and unreleased
struct file.

Additionally the fuse connection and with that the ring can
only get released when all io-uring commands are completed.

Completion is done with ring entries that are
a) in waiting state for new fuse requests - io_uring_cmd_done
is needed

b) already in userspace - io_uring_cmd_done through teardown
is not needed, the request can just get released. If fuse server
is still active and commits such a ring entry, fuse_uring_cmd()
already checks if the connection is active and then complete the
io-uring itself with -ENOTCONN. I.e. special handling is not
needed.

This scheme is basically represented by the ring entry state
FRRS_WAIT and FRRS_USERSPACE.

Entries in state:
- FRRS_INIT: No action needed, do not contribute to
ring->queue_refs yet
- All other states: Are currently processed by other tasks,
async teardown is needed and it has to wait for the two
states above. It could be also solved without an async
teardown task, but would require additional if conditions
in hot code paths. Also in my personal opinion the code
looks cleaner with async teardown.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: Add io-uring sqe commit and fetch support

This adds support for fuse request completion through ring SQEs
(FUSE_URING_CMD_COMMIT_AND_FETCH handling). After committing
the ring entry it becomes

fuse: Add io-uring sqe commit and fetch support

This adds support for fuse request completion through ring SQEs
(FUSE_URING_CMD_COMMIT_AND_FETCH handling). After committing
the ring entry it becomes available for new fuse requests.
Handling of requests through the ring (SQE/CQE handling)
is complete now.

Fuse request data are copied through the mmaped ring buffer,
there is no support for any zero copy yet.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...

fuse: {io-uring} Handle SQEs - register commands

This adds basic support for ring SQEs (with opcode=IORING_OP_URING_CMD).
For now only FUSE_IO_URING_CMD_REGISTER is handled to register queue
entries

fuse: {io-uring} Handle SQEs - register commands

This adds basic support for ring SQEs (with opcode=IORING_OP_URING_CMD).
For now only FUSE_IO_URING_CMD_REGISTER is handled to register queue
entries.

Signed-off-by: Bernd Schubert <[email protected]>
Reviewed-by: Pavel Begunkov <[email protected]> # io_uring
Reviewed-by: Luis Henriques <[email protected]>
Signed-off-by: Miklos Szeredi <[email protected]>

show more ...