Changes in cpumap.c

ed16b8a4 - bpf: cpumap: switch to napi_skb_cache_get_bulk()

Tue, 25 Feb 2025 17:17:48 +0000

bpf: cpumap: switch to napi_skb_cache_get_bulk()Now that cpumap uses GRO, which drops unused skb heads to the NAPIcache, use napi_skb_cache_get_bulk() to try to reuse cached entriesand lower MM layer pressure. Always disable the BH before checking andrunning the cpumap-pinned XDP prog and don't re-enable it in betweenthat and allocating an skb bulk, as we can access the NAPI caches onlyfrom the BH context.The better GRO aggregates packets, the less new skbs will be allocated.If an aggregated skb contains 16 frags, this means 15 skbs were returnedto the cache, so next 15 skbs will be built without allocating anything.The same trafficgen UDP GRO test now shows: GRO off GRO onthreaded GRO 2.3 4 Mppsthr bulk GRO 2.4 4.7 Mppsdiff +4 +17 %Comparing to the baseline cpumap:baseline 2.7 N/A Mppsthr bulk GRO 2.4 4.7 Mppsdiff -11 +74 %Tested-by: Daniel Xu Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Alexander Lobakin Signed-off-by: Paolo Abeni List of files: /linux-6.15/kernel/bpf/cpumap.c

57efe762 - bpf: cpumap: reuse skb array instead of a linked list to chain skbs

Tue, 25 Feb 2025 17:17:46 +0000

bpf: cpumap: reuse skb array instead of a linked list to chain skbscpumap still uses linked lists to store a list of skbs to pass to thestack. Now that we don't use listified Rx in favor ofnapi_gro_receive(), linked list is now an unneeded overhead.Inside the polling loop, we already have an array of skbs. Let's reuseit for skbs passed to cpumap (generic XDP) and keep there in case ofXDP_PASS when a program is installed to the map itself. Don't listregular xdp_frames after converting them to skbs as well; store themin the mentioned array (but *before* generic skbs as the latters havelower priority) and call gro_receive_skb() for each array element afterthey're done.Tested-by: Daniel Xu Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Alexander Lobakin Signed-off-by: Paolo Abeni List of files: /linux-6.15/kernel/bpf/cpumap.c

4f8ab26a - bpf: cpumap: switch to GRO from netif_receive_skb_list()

Tue, 25 Feb 2025 17:17:45 +0000

bpf: cpumap: switch to GRO from netif_receive_skb_list()cpumap has its own BH context based on kthread. It has a sane batchsize of 8 frames per one cycle.GRO can be used here on its own. Adjust cpumap calls to the upper stackto use GRO API instead of netif_receive_skb_list() which processes skbsby batches, but doesn't involve GRO layer at all.In plenty of tests, GRO performs better than listed receiving evengiven that it has to calculate full frame checksums on the CPU.As GRO passes the skbs to the upper stack in the batches of@gro_normal_batch, i.e. 8 by default, and skb->dev points to thedevice where the frame comes from, it is enough to disable GROnetdev feature on it to completely restore the original behaviour:untouched frames will be being bulked and passed to the upper stackby 8, as it was with netif_receive_skb_list().Tested-by: Daniel Xu Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Alexander Lobakin Signed-off-by: Paolo Abeni List of files: /linux-6.15/kernel/bpf/cpumap.c

56d95b0a - xdp: get rid of xdp_frame::mem.id

Wed, 11 Dec 2024 17:26:39 +0000

xdp: get rid of xdp_frame::mem.idInitially, xdp_frame::mem.id was used to search for the corresponding&page_pool to return the page correctly.However, after that struct page was extended to have a direct pointerto its PP (netmem has it as well), further keeping of this field makesno sense. xdp_return_frame_bulk() still used it to do a lookup, andthis leftover is now removed.Remove xdp_frame::mem and replace it with ::mem_type, as only memorytype still matters and we need to know it to be able to free the framecorrectly.As a cute side effect, we can now make every scalar field in &xdp_frameof 4 byte width, speeding up accesses to them.Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Alexander Lobakin Link: https://patch.msgid.link/20241211172649.761483-3-aleksander.lobakin@intel.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/kernel/bpf/cpumap.c

23dc9867 - bpf, cpumap: Move xdp:xdp_cpumap_kthread tracepoint before rcv

Fri, 06 Sep 2024 01:22:44 +0000

bpf, cpumap: Move xdp:xdp_cpumap_kthread tracepoint before rcvcpumap takes RX processing out of softirq and onto a separate kthread.Since the kthread needs to be scheduled in order to run (versus softirqwhich does not), we can theoretically experience extra latency if thesystem is under load and the scheduler is being unfair to us.Moving the tracepoint to before passing the skb list up the stack allowsusers to more accurately measure enqueue/dequeue latency introduced bycpumap via xdp:xdp_cpumap_enqueue and xdp:xdp_cpumap_kthread tracepoints.f9419f7bd7a5 ("bpf: cpumap add tracepoints") which added the tracepointsstates that the intent behind them was for general observability and fora feedback loop to see if the queues are being overwhelmed. This changedoes not mess with either of those use cases but rather adds a thirdone.Signed-off-by: Daniel Xu Signed-off-by: Daniel Borkmann Acked-by: Jesper Dangaard Brouer Link: https://lore.kernel.org/bpf/47615d5b5e302e4bd30220473779e98b492d47cd.1725585718.git.dxu@dxuuu.xyz List of files: /linux-6.15/kernel/bpf/cpumap.c

e3d69f58 - net: Move flush list retrieval to where it is used.

Fri, 28 Jun 2024 10:18:56 +0000

net: Move flush list retrieval to where it is used.The bpf_net_ctx_get_.*_flush_list() are used at the top of the function.This means the variable is always assigned even if unused. By moving thefunction to where it is used, it is possible to delay the initialisationuntil it is unavoidable.Not sure how much this gains in reality but by looking at bq_enqueue()(in devmap.c) gcc pushes one register less to the stack. \o/. Move flush list retrieval to where it is used.Signed-off-by: Sebastian Andrzej Siewior Acked-by: Jesper Dangaard Brouer Reviewed-by: Jakub Kicinski Signed-off-by: Paolo Abeni List of files: /linux-6.15/kernel/bpf/cpumap.c

d839a731 - net: Optimize xdp_do_flush() with bpf_net_context infos.

Fri, 28 Jun 2024 10:18:55 +0000

net: Optimize xdp_do_flush() with bpf_net_context infos.Every NIC driver utilizing XDP should invoke xdp_do_flush() afterprocessing all packages. With the introduction of the bpf_net_contextlogic the flush lists (for dev, CPU-map and xsk) are lazy initializedonly if used. However xdp_do_flush() tries to flush all three of them soall three lists are always initialized and the likely empty lists are"iterated".Without the usage of XDP but with CONFIG_DEBUG_NET the lists are alsoinitialized due to xdp_do_check_flushed().Jakub suggest to utilize the hints in bpf_net_context and avoid invokingthe flush function. This will also avoiding initializing the lists whichare otherwise unused.Introduce bpf_net_ctx_get_all_used_flush_lists() to return theindividual list if not-empty. Use the logic in xdp_do_flush() andxdp_do_check_flushed(). Remove the not needed .*_check_flush().Suggested-by: Jakub Kicinski Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Jakub Kicinski Signed-off-by: Paolo Abeni List of files: /linux-6.15/kernel/bpf/cpumap.c

3f9fe37d - net: Move per-CPU flush-lists to bpf_net_context on PREEMPT_RT.

Thu, 20 Jun 2024 13:22:05 +0000

net: Move per-CPU flush-lists to bpf_net_context on PREEMPT_RT.The per-CPU flush lists, which are accessed from within the NAPI callback(xdp_do_flush() for instance), are per-CPU. There are subject to thesame problem as struct bpf_redirect_info.Add the per-CPU lists cpu_map_flush_list, dev_map_flush_list andxskmap_map_flush_list to struct bpf_net_context. Add wrappers for theaccess. The lists initialized on first usage (similar tobpf_net_ctx_get_ri()).Cc: "Björn Töpel" Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Eduard Zingerman Cc: Hao Luo Cc: Jiri Olsa Cc: John Fastabend Cc: Jonathan Lemon Cc: KP Singh Cc: Maciej Fijalkowski Cc: Magnus Karlsson Cc: Martin KaFai Lau Cc: Song Liu Cc: Stanislav Fomichev Cc: Yonghong Song Acked-by: Jesper Dangaard Brouer Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20240620132727.660738-16-bigeasy@linutronix.deSigned-off-by: Jakub Kicinski List of files: /linux-6.15/kernel/bpf/cpumap.c

401cb7da - net: Reference bpf_redirect_info via task_struct on PREEMPT_RT.

Thu, 20 Jun 2024 13:22:04 +0000

net: Reference bpf_redirect_info via task_struct on PREEMPT_RT.The XDP redirect process is two staged:- bpf_prog_run_xdp() is invoked to run a eBPF program which inspects the packet and makes decisions. While doing that, the per-CPU variable bpf_redirect_info is used.- Afterwards xdp_do_redirect() is invoked and accesses bpf_redirect_info and it may also access other per-CPU variables like xskmap_flush_list.At the very end of the NAPI callback, xdp_do_flush() is invoked whichdoes not access bpf_redirect_info but will touch the individual per-CPUlists.The per-CPU variables are only used in the NAPI callback hence disablingbottom halves is the only protection mechanism. Users from preemptiblecontext (like cpu_map_kthread_run()) explicitly disable bottom halvesfor protections reasons.Without locking in local_bh_disable() on PREEMPT_RT this data structurerequires explicit locking.PREEMPT_RT has forced-threaded interrupts enabled and everyNAPI-callback runs in a thread. If each thread has its own datastructure then locking can be avoided.Create a struct bpf_net_context which contains struct bpf_redirect_info.Define the variable on stack, use bpf_net_ctx_set() to save a pointer toit, bpf_net_ctx_clear() removes it again.The bpf_net_ctx_set() may nest. For instance a function can be used fromwithin NET_RX_SOFTIRQ/ net_rx_action which uses bpf_net_ctx_set() andNET_TX_SOFTIRQ which does not. Therefore only the first invocationsupdates the pointer.Use bpf_net_ctx_get_ri() as a wrapper to retrieve the current structbpf_redirect_info. The returned data structure is zero initialized toensure nothing is leaked from stack. This is done on first usage of thestruct. bpf_net_ctx_set() sets bpf_redirect_info::kern_flags to 0 tonote that initialisation is required. First invocation ofbpf_net_ctx_get_ri() will memset() the data structure and updatebpf_redirect_info::kern_flags.bpf_redirect_info::nh is excluded from memset because it is only usedonce BPF_F_NEIGH is set which also sets the nh member. The kern_flags ismoved past nh to exclude it from memset.The pointer to bpf_net_context is saved task's task_struct. Usingalways the bpf_net_context approach has the advantage that there isalmost zero differences between PREEMPT_RT and non-PREEMPT_RT builds.Cc: Andrii Nakryiko Cc: Eduard Zingerman Cc: Hao Luo Cc: Jiri Olsa Cc: John Fastabend Cc: KP Singh Cc: Martin KaFai Lau Cc: Song Liu Cc: Stanislav Fomichev Cc: Yonghong Song Acked-by: Alexei Starovoitov Acked-by: Jesper Dangaard Brouer Reviewed-by: Toke Høiland-Jørgensen Signed-off-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20240620132727.660738-15-bigeasy@linutronix.deSigned-off-by: Jakub Kicinski List of files: /linux-6.15/kernel/bpf/cpumap.c

00bf6312 - bpf: report RCU QS in cpumap kthread

Tue, 19 Mar 2024 20:44:40 +0000

bpf: report RCU QS in cpumap kthreadWhen there are heavy load, cpumap kernel threads can be busy pollingpackets from redirect queues and block out RCU tasks from reachingquiescent states. It is insufficient to just call cond_resched() in suchcontext. Periodically raise a consolidated RCU QS before cond_reschedfixes the problem.Fixes: 6710e1126934 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")Reviewed-by: Jesper Dangaard Brouer Signed-off-by: Yan Zhai Acked-by: Paul E. McKenney Acked-by: Jesper Dangaard Brouer Link: https://lore.kernel.org/r/c17b9f1517e19d813da3ede5ed33ee18496bb5d8.1710877680.git.yan@cloudflare.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/kernel/bpf/cpumap.c

aa70d2d1 - net: move skbuff_cache(s) to net_hotdata

Wed, 06 Mar 2024 16:00:23 +0000

net: move skbuff_cache(s) to net_hotdataskbuff_cache, skbuff_fclone_cache and skb_small_head_cacheare used in rx/tx fast paths.Move them to net_hotdata for better cache locality.Signed-off-by: Eric Dumazet Acked-by: Soheil Hassas Yeganeh Reviewed-by: David Ahern Link: https://lore.kernel.org/r/20240306160031.874438-11-edumazet@google.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/kernel/bpf/cpumap.c

2487007a - cpumap: Zero-initialise xdp_rxq_info struct before running XDP program

Tue, 05 Mar 2024 21:31:32 +0000

cpumap: Zero-initialise xdp_rxq_info struct before running XDP programWhen running an XDP program that is attached to a cpumap entry, we don'tinitialise the xdp_rxq_info data structure being used in the xdp_buffthat backs the XDP program invocation. Tobias noticed that this leads torandom values being returned as the xdp_md->rx_queue_index value for XDPprograms running in a cpumap.This means we're basically returning the contents of the uninitialisedmemory, which is bad. Fix this by zero-initialising the rxq datastructure before running the XDP program.Fixes: 9216477449f3 ("bpf: cpumap: Add the possibility to attach an eBPF program to cpumap")Reported-by: Tobias Böhm Signed-off-by: Toke Høiland-Jørgensen Link: https://lore.kernel.org/r/20240305213132.11955-1-toke@redhat.comSigned-off-by: Martin KaFai Lau List of files: /linux-6.15/kernel/bpf/cpumap.c

9a675ba5 - net, bpf: Add a warning if NAPI cb missed xdp_do_flush().

Mon, 16 Oct 2023 12:57:38 +0000

net, bpf: Add a warning if NAPI cb missed xdp_do_flush().A few drivers were missing a xdp_do_flush() invocation afterXDP_REDIRECT.Add three helper functions each for one of the per-CPU lists. Returntrue if the per-CPU list is non-empty and flush the list.Add xdp_do_check_flushed() which invokes each helper functions andcreates a warning if one of the functions had a non-empty list.Hide everything behind CONFIG_DEBUG_NET.Suggested-by: Jesper Dangaard Brouer Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Daniel Borkmann Reviewed-by: Toke Høiland-Jørgensen Acked-by: Jakub Kicinski Acked-by: John Fastabend Link: https://lore.kernel.org/bpf/20231016125738.Yt79p1uF@linutronix.de List of files: /linux-6.15/kernel/bpf/cpumap.c

c2e42ddf - bpf, cpumask: Clean up bpf_cpu_map_entry directly in cpu_map_free

Wed, 16 Aug 2023 04:59:58 +0000

bpf, cpumask: Clean up bpf_cpu_map_entry directly in cpu_map_freeAfter synchronous_rcu(), both the dettached XDP program andxdp_do_flush() are completed, and the only user of bpf_cpu_map_entrywill be cpu_map_kthread_run(), so instead of calling__cpu_map_entry_replace() to stop kthread and cleanup entry after a RCUgrace period, do these things directly.Signed-off-by: Hou Tao Reviewed-by: Toke Høiland-Jørgensen Link: https://lore.kernel.org/r/20230816045959.358059-3-houtao@huaweicloud.comSigned-off-by: Alexei Starovoitov List of files: /linux-6.15/kernel/bpf/cpumap.c

8f8500a2 - bpf, cpumap: Use queue_rcu_work() to remove unnecessary rcu_barrier()

Wed, 16 Aug 2023 04:59:57 +0000

bpf, cpumap: Use queue_rcu_work() to remove unnecessary rcu_barrier()As for now __cpu_map_entry_replace() uses call_rcu() to wait for theinflight xdp program to exit the RCU read critical section, and thenlaunch kworker cpu_map_kthread_stop() to call kthread_stop() to flushall pending xdp frames or skbs.But it is unnecessary to use rcu_barrier() in cpu_map_kthread_stop() towait for the completion of __cpu_map_entry_free(), because rcu_barrier()will wait for all pending RCU callbacks and cpu_map_kthread_stop() onlyneeds to wait for the completion of a specific __cpu_map_entry_free().So use queue_rcu_work() to replace call_rcu(), schedule_work() andrcu_barrier(). queue_rcu_work() will queue a __cpu_map_entry_free()kworker after a RCU grace period. Because __cpu_map_entry_free() isrunning in a kworker context, so it is OK to do all of these freeingprocedures include kthread_stop() in it.After the update, there is no need to do reference-counting forbpf_cpu_map_entry, because bpf_cpu_map_entry is freed directly in__cpu_map_entry_free(), so just remove it.Signed-off-by: Hou Tao Reviewed-by: Toke Høiland-Jørgensen Link: https://lore.kernel.org/r/20230816045959.358059-2-houtao@huaweicloud.comSigned-off-by: Alexei Starovoitov List of files: /linux-6.15/kernel/bpf/cpumap.c

2d20bfc3 - bpf, cpumap: Remove unused cmap field from bpf_cpu_map_entry

Fri, 28 Jul 2023 01:49:41 +0000

bpf, cpumap: Remove unused cmap field from bpf_cpu_map_entrySince commit cdfafe98cabe ("xdp: Make cpumap flush_list common for allmap instances"), cmap is no longer used, so just remove it.Signed-off-by: Hou Tao Acked-by: Jesper Dangaard Brouer Acked-by: Jiri Olsa Link: https://lore.kernel.org/r/20230728014942.892272-2-houtao@huaweicloud.comSigned-off-by: Martin KaFai Lau List of files: /linux-6.15/kernel/bpf/cpumap.c

7c62b75c - bpf, cpumap: Handle skb as well when clean up ptr_ring

Sat, 29 Jul 2023 09:51:07 +0000

bpf, cpumap: Handle skb as well when clean up ptr_ringThe following warning was reported when running xdp_redirect_cpu withboth skb-mode and stress-mode enabled: ------------[ cut here ]------------ Incorrect XDP memory type (-2128176192) usage WARNING: CPU: 7 PID: 1442 at net/core/xdp.c:405 Modules linked in: CPU: 7 PID: 1442 Comm: kworker/7:0 Tainted: G 6.5.0-rc2+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) Workqueue: events __cpu_map_entry_free RIP: 0010:__xdp_return+0x1e4/0x4a0 ...... Call Trace: ? show_regs+0x65/0x70 ? __warn+0xa5/0x240 ? __xdp_return+0x1e4/0x4a0 ...... xdp_return_frame+0x4d/0x150 __cpu_map_entry_free+0xf9/0x230 process_one_work+0x6b0/0xb80 worker_thread+0x96/0x720 kthread+0x1a5/0x1f0 ret_from_fork+0x3a/0x70 ret_from_fork_asm+0x1b/0x30 The reason for the warning is twofold. One is due to the kthreadcpu_map_kthread_run() is stopped prematurely. Another one is__cpu_map_ring_cleanup() doesn't handle skb mode and treats skbs inptr_ring as XDP frames.Prematurely-stopped kthread will be fixed by the preceding patch andptr_ring will be empty when __cpu_map_ring_cleanup() is called. Butas the comments in __cpu_map_ring_cleanup() said, handling and freeingskbs in ptr_ring as well to "catch any broken behaviour gracefully".Fixes: 11941f8a8536 ("bpf: cpumap: Implement generic cpumap")Signed-off-by: Hou Tao Acked-by: Jesper Dangaard Brouer Link: https://lore.kernel.org/r/20230729095107.1722450-3-houtao@huaweicloud.comSigned-off-by: Martin KaFai Lau List of files: /linux-6.15/kernel/bpf/cpumap.c

640a6045 - bpf, cpumap: Make sure kthread is running before map update returns

Sat, 29 Jul 2023 09:51:06 +0000

bpf, cpumap: Make sure kthread is running before map update returnsThe following warning was reported when running stress-mode enabledxdp_redirect_cpu with some RT threads: ------------[ cut here ]------------ WARNING: CPU: 4 PID: 65 at kernel/bpf/cpumap.c:135 CPU: 4 PID: 65 Comm: kworker/4:1 Not tainted 6.5.0-rc2+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) Workqueue: events cpu_map_kthread_stop RIP: 0010:put_cpu_map_entry+0xda/0x220 ...... Call Trace: ? show_regs+0x65/0x70 ? __warn+0xa5/0x240 ...... ? put_cpu_map_entry+0xda/0x220 cpu_map_kthread_stop+0x41/0x60 process_one_work+0x6b0/0xb80 worker_thread+0x96/0x720 kthread+0x1a5/0x1f0 ret_from_fork+0x3a/0x70 ret_from_fork_asm+0x1b/0x30 The root cause is the same as commit 436901649731 ("bpf: cpumap: Fix memoryleak in cpu_map_update_elem"). The kthread is stopped prematurely bykthread_stop() in cpu_map_kthread_stop(), and kthread() doesn't callcpu_map_kthread_run() at all but XDP program has already queued someframes or skbs into ptr_ring. So when __cpu_map_ring_cleanup() checksthe ptr_ring, it will find it was not emptied and report a warning.An alternative fix is to use __cpu_map_ring_cleanup() to drop thesepending frames or skbs when kthread_stop() returns -EINTR, but it mayconfuse the user, because these frames or skbs have been handledcorrectly by XDP program. So instead of dropping these frames or skbs,just make sure the per-cpu kthread is running before__cpu_map_entry_alloc() returns.After apply the fix, the error handle for kthread_stop() will beunnecessary because it will always return 0, so just remove it.Fixes: 6710e1126934 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")Signed-off-by: Hou Tao Reviewed-by: Pu Lehui Acked-by: Jesper Dangaard Brouer Link: https://lore.kernel.org/r/20230729095107.1722450-2-houtao@huaweicloud.comSigned-off-by: Martin KaFai Lau List of files: /linux-6.15/kernel/bpf/cpumap.c

43690164 - bpf: cpumap: Fix memory leak in cpu_map_update_elem

Tue, 11 Jul 2023 11:58:48 +0000

bpf: cpumap: Fix memory leak in cpu_map_update_elemSyzkaller reported a memory leak as follows:BUG: memory leakunreferenced object 0xff110001198ef748 (size 192): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 32 bytes): 00 00 00 00 4a 19 00 00 80 ad e3 e4 fe ff c0 00 ....J........... 00 b2 d3 0c 01 00 11 ff 28 f5 8e 19 01 00 11 ff ........(....... backtrace: [] __cpu_map_entry_alloc+0xf7/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6BUG: memory leakunreferenced object 0xff110001198ef528 (size 192): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [] __cpu_map_entry_alloc+0x260/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6BUG: memory leakunreferenced object 0xff1100010fd93d68 (size 8): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 8 bytes): 00 00 00 00 00 00 00 00 ........ backtrace: [] kvmalloc_node+0x11e/0x170 [] __cpu_map_entry_alloc+0x2f0/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6In the cpu_map_update_elem flow, when kthread_stop is called beforecalling the threadfn of rcpu->kthread, since the KTHREAD_SHOULD_STOP bitof kthread has been set by kthread_stop, the threadfn of rcpu->kthreadwill never be executed, and rcpu->refcnt will never be 0, which willlead to the allocated rcpu, rcpu->queue and rcpu->queue->queue cannot bereleased.Calling kthread_stop before executing kthread's threadfn will return-EINTR. We can complete the release of memory resources in this state.Fixes: 6710e1126934 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")Signed-off-by: Pu Lehui Acked-by: Jesper Dangaard Brouer Acked-by: Hou Tao Link: https://lore.kernel.org/r/20230711115848.2701559-1-pulehui@huaweicloud.comSigned-off-by: Alexei Starovoitov List of files: /linux-6.15/kernel/bpf/cpumap.c

6c3eba1c - bpf: Centralize permissions checks for all BPF map types

Tue, 13 Jun 2023 22:35:32 +0000

bpf: Centralize permissions checks for all BPF map typesThis allows to do more centralized decisions later on, and generallymakes it very explicit which maps are privileged and which are not(e.g., LRU_HASH and LRU_PERCPU_HASH, which are privileged HASH variants,as opposed to unprivileged HASH and HASH_PERCPU; now this is explicitand easy to verify).Signed-off-by: Andrii Nakryiko Signed-off-by: Daniel Borkmann Acked-by: Stanislav Fomichev Link: https://lore.kernel.org/bpf/20230613223533.3689589-4-andrii@kernel.org List of files: /linux-6.15/kernel/bpf/cpumap.c