Changes in Makefile

0fb3f560 - io_uring/epoll: remove CONFIG_EPOLL guards

Fri, 31 Jan 2025 21:19:11 +0000

io_uring/epoll: remove CONFIG_EPOLL guardsJust have the Makefile add the object if epoll is enabled, then it'snot necessary to guard the entire epoll.c file inside an CONFIG_EPOLLifdef.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

6f377873 - io_uring/zcrx: add interface queue and refill queue

Sat, 15 Feb 2025 00:09:36 +0000

io_uring/zcrx: add interface queue and refill queueAdd a new object called an interface queue (ifq) that represents a netrx queue that has been configured for zero copy. Each ifq is registeredusing a new registration opcode IORING_REGISTER_ZCRX_IFQ.The refill queue is allocated by the kernel and mapped by userspaceusing a new offset IORING_OFF_RQ_RING, in a similar fashion to the mainSQ/CQ. It is used by userspace to return buffers that it is done with,which will then be re-used by the netdev again.The main CQ ring is used to notify userspace of received data by usingthe upper 16 bytes of a big CQE as a new struct io_uring_zcrx_cqe. Eachentry contains the offset + len to the data.For now, each io_uring instance only has a single ifq.Reviewed-by: Jens Axboe Signed-off-by: David Wei Acked-by: Jakub Kicinski Link: https://lore.kernel.org/r/20250215000947.789731-2-dw@davidwei.ukSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

d19af0e9 - io_uring: add alloc_cache.c

Tue, 28 Jan 2025 20:56:11 +0000

io_uring: add alloc_cache.cAvoid inlining all and everything from alloc_cache.h and move cold bitsinto a new file.Signed-off-by: Pavel Begunkov Reviewed-by: Gabriel Krisman Bertazi Link: https://lore.kernel.org/r/06984c6cd58e703f7cfae5ab3067912f9f635a06.1738087204.git.asml.silence@gmail.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

1802656e - io_uring: add GCOV_PROFILE_URING Kconfig option

Fri, 30 Aug 2024 16:52:02 +0000

io_uring: add GCOV_PROFILE_URING Kconfig optionIf GCOV is enabled and this option is set, it enables code coverageprofiling of the io_uring subsystem. Only use this for test purposes,as it will impact the runtime performance.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

200f3abd - io_uring/eventfd: move eventfd handling to separate file

Mon, 03 Jun 2024 17:51:19 +0000

io_uring/eventfd: move eventfd handling to separate fileThis is pretty nicely abstracted already, but let's move it to a separatefile rather than have it in the main io_uring file. With that, we canalso move the io_ev_fd struct and enum out of global scope.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

f15ed8b4 - io_uring: move mapping/allocation helpers to a separate file

Wed, 27 Mar 2024 20:59:09 +0000

io_uring: move mapping/allocation helpers to a separate fileMove the related code from io_uring.c into memmap.c. No functionalchanges in this patch, just cleaning it up a bit now that the fulltransition is done.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

77a1cd5e - io_uring: re-arrange Makefile order

Wed, 27 Mar 2024 04:13:41 +0000

io_uring: re-arrange Makefile orderThe object list is a bit of a mess, with core and opcode files mixed in.Re-arrange it so that we have the core bits first, and then opcodespecific files after that.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

8d0c12a8 - io-uring: add napi busy poll support

Thu, 08 Jun 2023 16:38:36 +0000

io-uring: add napi busy poll supportThis adds the napi busy polling support in io_uring.c. It adds a newnapi_list to the io_ring_ctx structure. This list contains the list ofnapi_id's that are currently enabled for busy polling. The list issynchronized by the new napi_lock spin lock. The current default napibusy polling time is stored in napi_busy_poll_to. If napi busy pollingis not enabled, the value is 0.In addition there is also a hash table. The hash table store the napiid and the pointer to the above list nodes. The hash table is used tospeed up the lookup to the list elements. The hash table is synchronizedwith rcu.The NAPI_TIMEOUT is stored as a timeout to make sure that the time anapi entry is stored in the napi list is limited.The busy poll timeout is also stored as part of the io_wait_queue. Thisis necessary as for sq polling the poll interval needs to be adjustedand the napi callback allows only to pass in one value.This has been tested with two simple programs from the liburing libraryrepository: the napi client and the napi server program. The clientsends a request, which has a timestamp in its payload and the serverreplies with the same payload. The client calculates the roundtrip timeand stores it to calculate the results.The client is running on host1 and the server is running on host 2 (inthe same rack). The measured times below are roundtrip times. They areaverage times over 5 runs each. Each run measures 1 million roundtrips. no rx coal rx coal: frames=88,usecs=33Default 57us 56usclient_poll=100us 47us 46usserver_poll=100us 51us 46usclient_poll=100us+ 40us 40usserver_poll=100usclient_poll=100us+ 41us 39usserver_poll=100us+prefer napi busy poll on clientclient_poll=100us+ 41us 39usserver_poll=100us+prefer napi busy poll on serverclient_poll=100us+ 41us 39usserver_poll=100us+prefer napi busy poll on client + serverSigned-off-by: Stefan Roesch Suggested-by: Olivier Langlois Acked-by: Jakub Kicinski Link: https://lore.kernel.org/r/20230608163839.2891748-5-shr@devkernel.ioSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

b4bb1900 - io_uring: add support for ftruncate

Fri, 02 Feb 2024 12:17:24 +0000

io_uring: add support for ftruncateAdds support for doing truncate through io_uring, eliminatingthe need for applications to roll their own thread pool or offloadmechanism to be able to do non-blocking truncates.Signed-off-by: Tony Solomonik Link: https://lore.kernel.org/r/20240202121724.17461-3-tony.solomonik@gmail.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

c4320315 - io_uring/register: move io_uring_register(2) related code to register.c

Tue, 19 Dec 2023 15:54:20 +0000

io_uring/register: move io_uring_register(2) related code to register.cMost of this code is basically self contained, move it out of the coreio_uring file to bring a bit more separation to the registration relatedbits. This moves another ~10% of the code into register.c.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

194bb58c - io_uring: add support for futex wake and wait

Thu, 08 Jun 2023 17:57:40 +0000

io_uring: add support for futex wake and waitAdd support for FUTEX_WAKE/WAIT primitives.IORING_OP_FUTEX_WAKE is mix of FUTEX_WAKE and FUTEX_WAKE_BITSET, asit does support passing in a bitset.Similary, IORING_OP_FUTEX_WAIT is a mix of FUTEX_WAIT andFUTEX_WAIT_BITSET.For both of them, they are using the futex2 interface.FUTEX_WAKE is straight forward, as those can always be done directly fromthe io_uring submission without needing async handling. For FUTEX_WAIT,things are a bit more complicated. If the futex isn't ready, then werely on a callback via futex_queue->wake() when someone wakes up thefutex. From that calback, we queue up task_work with the original task,which will post a CQE and wake it, if necessary.Cancelations are supported, both from the application point-of-view,but also to be able to cancel pending waits if the ring exits beforeall events have occurred. The return value of futex_unqueue() is usedto gate who wins the potential race between cancelation and futexwakeups. Whomever gets a 'ret == 1' return from that claims ownershipof the io_uring futex request.This is just the barebones wait/wake support. PI or REQUEUE support isnot added at this point, unclear if we might look into that later.Likewise, explicit timeouts are not supported either. It is expectedthat users that need timeouts would do so via the usual io_uringmechanism to do that using linked timeouts.The SQE format is as follows:`addr` Address of futex`fd` futex2(2) FUTEX2_* flags`futex_flags` io_uring specific command flags. None valid now.`addr2` Value of futex`addr3` Mask to wake/waitAcked-by: Peter Zijlstra (Intel) Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

f31ecf67 - io_uring: add IORING_OP_WAITID support

Mon, 10 Jul 2023 22:14:37 +0000

io_uring: add IORING_OP_WAITID supportThis adds support for an async version of waitid(2), in a fully asyncversion. If an event isn't immediately available, wait for a callbackto trigger a retry.The format of the sqe is as follows:sqe->len The 'which', the idtype being queried/waited for.sqe->fd The 'pid' (or id) being waited for.sqe->file_index The 'options' being set.sqe->addr2 A pointer to siginfo_t, if any, being filled in.buf_index, add3, and waitid_flags are reserved/unused for now.waitid_flags will be used for options for this request type. Oneinteresting use case may be to add multi-shot support, so that therequest stays armed and posts a notification every time a monitoredprocess state change occurs.Note that this does not support rusage, on Arnd's recommendation.See the waitid(2) man page for details on the arguments.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

eb42cebb - io_uring: add zc notification infrastructure

Tue, 12 Jul 2022 20:52:38 +0000

io_uring: add zc notification infrastructureAdd internal part of send zerocopy notifications. There are two mainstructures, the first one is struct io_notif, which carries insidestruct ubuf_info and maps 1:1 to it. io_uring will be binding a numberof zerocopy send requests to it and ask to complete (aka flush) it. Whenflushed and all attached requests and skbs complete, it'll generate oneand only one CQE. There are intended to be passed into the network layeras struct msghdr::msg_ubuf.The second concept is notification slots. The userspace will be able toregister an array of slots and subsequently addressing them by the indexin the array. Slots are independent of each other. Each slot can haveonly one notifier at a time (called active notifier) but many notifiersduring the lifetime. When active, a notifier not going to post anycompletion but the userspace can attach requests to it by specifyingthe corresponding slot while issueing send zc requests. Eventually, theuserspace will want to "flush" the notifier losing any way to attachnew requests to it, however it can use the next atomatically addednotifier of this slot or of any other slot.When the network layer is done with all enqueued skbs attached to anotifier and doesn't need the specified in them user data, the flushednotifier will post a CQE.Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/3ecf54c31a85762bf679b0a432c9f43ecf7e61cc.1657643355.git.asml.silence@gmail.comSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

d9b57aa3 - io_uring: move opcode table to opdef.c

Wed, 15 Jun 2022 22:27:42 +0000

io_uring: move opcode table to opdef.cWe already have the declarations in opdef.h, move the rest into its ownfile rather than in the main io_uring.c file.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

f3b44f92 - io_uring: move read/write related opcodes to its own file

Mon, 13 Jun 2022 13:27:03 +0000

io_uring: move read/write related opcodes to its own fileSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

73572984 - io_uring: move rsrc related data, core, and commands

Mon, 13 Jun 2022 13:12:45 +0000

io_uring: move rsrc related data, core, and commandsSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

3b77495a - io_uring: split provided buffers handling into its own file

Mon, 13 Jun 2022 13:07:23 +0000

io_uring: split provided buffers handling into its own fileMove both the opcodes related to it, and the internals code dealing withit.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

7aaff708 - io_uring: move cancelation into its own file

Thu, 26 May 2022 02:36:47 +0000

io_uring: move cancelation into its own fileThis also helps cleanup the io_uring.h cancel parts, as we can makethings static in the cancel.c file, mostly.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

329061d3 - io_uring: move poll handling into its own file

Thu, 26 May 2022 02:31:09 +0000

io_uring: move poll handling into its own fileAdd a io_poll_issue() rather than export the general task_work lockingand io_issue_sqe(), and put the io_op_defs definition and structure intoa separate header file so that poll can use it.Signed-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile

c9f06aa7 - io_uring: move io_uring_task (tctx) helpers into its own file

Wed, 25 May 2022 17:01:04 +0000

io_uring: move io_uring_task (tctx) helpers into its own fileSigned-off-by: Jens Axboe List of files: /linux-6.15/io_uring/Makefile