Changes in tctx.c

7eb75ce7 - io_uring/tctx: work around xa_store() allocation error issue

Fri, 29 Nov 2024 14:20:28 +0000

io_uring/tctx: work around xa_store() allocation error issuesyzbot triggered the following WARN_ON:WARNING: CPU: 0 PID: 16 at io_uring/tctx.c:51 __io_uring_free+0xfa/0x140 io_uring/tctx.c:51which is theWARN_ON_ONCE(!xa_empty(&tctx->xa));sanity check in __io_uring_free() when a io_uring_task is going throughits final put. The syzbot test case includes injecting memory allocationfailures, and it very much looks like xa_store() can fail one of itsmemory allocations and end up with ->head being non-NULL even though noentries exist in the xarray.Until this issue gets sorted out, work around it by attempting toiterate entries in our xarray, and WARN_ON_ONCE() if one is found.Reported-by: syzbot+cc36d44ec9f368e443d3@syzkaller.appspotmail.comLink: https://lore.kernel.org/io-uring/673c1643.050a0220.87769.0066.GAE@google.com/Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

b6f58a3f - io_uring: move struct io_kiocb from task_struct to io_uring_task

Sun, 03 Nov 2024 17:23:38 +0000

io_uring: move struct io_kiocb from task_struct to io_uring_taskRather than store the task_struct itself in struct io_kiocb, storethe io_uring specific task_struct. The life times are the same in termsof io_uring, and this avoids doing some dereferences through thetask_struct. For the hot path of putting local task references, we canderef req->tctx instead, which we'll need anyway in that functionregardless of whether it's local or remote references.This is mostly straight forward, except the original task PF_EXITINGcheck needs a bit of tweaking. task_work is _always_ run from theoriginating task, except in the fallback case, where it's run from akernel thread. Replace the potentially racy (in case of fallback work)checks for req->task->flags with current->flags. It's either the stillthe original task, in which case PF_EXITING will be sane, or it hasPF_KTHREAD set, in which case it's fallback work. Both cases shouldprevent moving forward with the given request.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

6e76ac59 - io_uring: Add io_uring_setup flag to pre-register ring fd and never install it

Fri, 28 Apr 2023 16:40:30 +0000

io_uring: Add io_uring_setup flag to pre-register ring fd and never install itWith IORING_REGISTER_USE_REGISTERED_RING, an application can registerthe ring fd and use it via registered index rather than installed fd.This allows using a registered ring for everything *except* the initialmmap.With IORING_SETUP_NO_MMAP, io_uring_setup uses buffers allocated by theuser, rather than requiring a subsequent mmap.The combination of the two allows a user to operate *entirely* via aregistered ring fd, making it unnecessary to ever install the fd in thefirst place. So, add a flag IORING_SETUP_REGISTERED_FD_ONLY to makeio_uring_setup register the fd and return a registered index, withoutinstalling the fd.This allows an application to avoid touching the fd table at all, andallows a library to never even momentarily install a file descriptor.This splits out an io_ring_add_registered_file helper fromio_ring_add_registered_fd, for use by io_uring_setup.Signed-off-by: Josh Triplett Link: https://lore.kernel.org/r/bc8f431bada371c183b95a83399628b605e978a3.1682699803.git.josh@joshtriplett.orgSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

8d664282 - io_uring: rename 'in_idle' to 'in_cancel'

Fri, 17 Feb 2023 15:27:23 +0000

io_uring: rename 'in_idle' to 'in_cancel'This better describes what it does - it's incremented when the task iscurrently undergoing a cancelation operation, due to exiting or exec'ing.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

4add705e - io_uring: remove io_register_submitter

Mon, 26 Sep 2022 17:09:27 +0000

io_uring: remove io_register_submitterthis is no longer needed, as submitter_task is set at creation time.Signed-off-by: Dylan Yudaken Fixes: 97bbdc06a444 ("io_uring: add IORING_SETUP_SINGLE_ISSUER")Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

97c96e9f - io_uring: simplify __io_uring_add_tctx_node

Mon, 26 Sep 2022 17:09:26 +0000

io_uring: simplify __io_uring_add_tctx_nodeRemove submitter parameter from __io_uring_add_tctx_node.It was only called from one place, and we can do that logic in that oneplace.Signed-off-by: Dylan Yudaken Fixes: 97bbdc06a444 ("io_uring: add IORING_SETUP_SINGLE_ISSUER")Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

f88262e6 - io_uring: lockless task list

Wed, 22 Jun 2022 13:40:23 +0000

io_uring: lockless task listWith networking use cases we see contention on the spinlock used toprotect the task_list when multiple threads try and add completions at once.Instead we can use a lockless list, and assume that the first caller toadd to the list is responsible for kicking off task work.Signed-off-by: Dylan Yudaken Link: https://lore.kernel.org/r/20220622134028.2013417-4-dylany@fb.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

ed5ccb3b - io_uring: remove priority tw list optimisation

Wed, 22 Jun 2022 13:40:21 +0000

io_uring: remove priority tw list optimisationThis optimisation has some built in assumptions that make it easy tointroduce bugs. It also does not have clear wins that make it worth keeping.Signed-off-by: Dylan Yudaken Link: https://lore.kernel.org/r/20220622134028.2013417-2-dylany@fb.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

27a9d66f - io_uring: kill extra io_uring_types.h includes

Thu, 16 Jun 2022 12:57:18 +0000

io_uring: kill extra io_uring_types.h includesio_uring/io_uring.h already includes io_uring_types.h, no need toinclude it every time. Kill it in a bunch of places, it prepares us forfollowing patches.Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/94d8c943fbe0ef949981c508ddcee7fc1c18850f.1655384063.git.asml.silence@gmail.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

97bbdc06 - io_uring: add IORING_SETUP_SINGLE_ISSUER

Thu, 16 Jun 2022 09:22:08 +0000

io_uring: add IORING_SETUP_SINGLE_ISSUERAdd a new IORING_SETUP_SINGLE_ISSUER flag and the userspace visible partof it, i.e. put limitations of submitters. Also, don't allow it togetherwith IOPOLL as we're not going to put it to good use.Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/4bcc41ee467fdf04c8aab8baf6ce3ba21858c3d4.1655371007.git.asml.silence@gmail.comReviewed-by: Hao Xu Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

6a02e4be - io_uring: inline ->registered_rings

Wed, 15 Jun 2022 16:33:53 +0000

io_uring: inline ->registered_ringsThere can be only 16 registered rings, no need to allocate an array forthem separately but store it in tctx.Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/495f0b953c87994dd9e13de2134019054fa5830d.1655310733.git.asml.silence@gmail.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c

c9f06aa7 - io_uring: move io_uring_task (tctx) helpers into its own file

Wed, 25 May 2022 17:01:04 +0000

io_uring: move io_uring_task (tctx) helpers into its own fileSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/tctx.c