Changes in device.c

8fa7292f - treewide: Switch/rename to timer_delete[_sync]()

Sat, 05 Apr 2025 08:17:26 +0000

treewide: Switch/rename to timer_delete[_sync]()timer_delete[_sync]() replaces del_timer[_sync](). Convert the whole treeover and remove the historical wrapper inlines.Conversion was done with coccinelle plus manual fixups where necessary.Signed-off-by: Thomas Gleixner Signed-off-by: Ingo Molnar List of files: /linux-6.15/drivers/net/wireguard/device.c

cf517ac1 - net: Use link/peer netns in newlink() of rtnl_link_ops

Wed, 19 Feb 2025 12:50:29 +0000

net: Use link/peer netns in newlink() of rtnl_link_opsAdd two helper functions - rtnl_newlink_link_net() andrtnl_newlink_peer_net() for netns fallback logic. Peer netns falls backto link netns, and link netns falls back to source netns.Convert the use of params->net in netdevice drivers to one of the helperfunctions for clarity.Signed-off-by: Xiao Liang Reviewed-by: Kuniyuki Iwashima Link: https://patch.msgid.link/20250219125039.18024-4-shaw.leon@gmail.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

69c7be1b - rtnetlink: Pack newlink() params into struct

Wed, 19 Feb 2025 12:50:28 +0000

rtnetlink: Pack newlink() params into structThere are 4 net namespaces involved when creating links: - source netns - where the netlink socket resides, - target netns - where to put the device being created, - link netns - netns associated with the device (backend), - peer netns - netns of peer device.Currently, two nets are passed to newlink() callback - "src_net"parameter and "dev_net" (implicitly in net_device). They are set asfollows, depending on netlink attributes in the request. +------------+-------------------+---------+---------+ | peer netns | IFLA_LINK_NETNSID | src_net | dev_net | +------------+-------------------+---------+---------+ | | absent | source | target | | absent +-------------------+---------+---------+ | | present | link | link | +------------+-------------------+---------+---------+ | | absent | peer | target | | present +-------------------+---------+---------+ | | present | peer | link | +------------+-------------------+---------+---------+When IFLA_LINK_NETNSID is present, the device is created in link netnsfirst and then moved to target netns. This has some side effects,including extra ifindex allocation, ifname validation and link events.These could be avoided if we create it in target netns fromthe beginning.On the other hand, the meaning of src_net parameter is ambiguous. Itvaries depending on how parameters are passed. It is the effectivelink (or peer netns) by design, but some drivers ignore it and usedev_net instead.To provide more netns context for drivers, this patch packs existingnewlink() parameters, along with the source netns, link netns and peernetns, into a struct. The old "src_net" is renamed to "net" to avoidconfusion with real source netns, and will be deprecated later. The useof src_net are converted to params->net trivially.Signed-off-by: Xiao Liang Reviewed-by: Kuniyuki Iwashima Link: https://patch.msgid.link/20250219125039.18024-3-shaw.leon@gmail.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

06a34f7d - wireguard: device: support big tcp GSO

Sun, 17 Nov 2024 21:20:30 +0000

wireguard: device: support big tcp GSOAdvertise GSO_MAX_SIZE as TSO max size in order support BIG TCP for wireguard.This helps to improve wireguard performance a bit when enabled as it allowswireguard to aggregate larger skbs in wg_packet_consume_data_done() vianapi_gro_receive(), but also allows the stack to build larger skbs on xmitwhere the driver then segments them before encryption inside wg_xmit().We've seen a 15% improvement in TCP stream performance.Signed-off-by: Daniel Borkmann Signed-off-by: Jason A. Donenfeld Link: https://patch.msgid.link/20241117212030.629159-5-Jason@zx2c4.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

2c862914 - wireguard: device: omit unnecessary memset of netdev private data

Sun, 17 Nov 2024 21:20:27 +0000

wireguard: device: omit unnecessary memset of netdev private dataThe memory for netdev_priv is allocated using kvzalloc inalloc_netdev_mqs before rtnl_link_ops->setup is called so there is noneed to zero it again in wg_setup.Signed-off-by: Tobias Klauser Reviewed-by: Simon Horman Signed-off-by: Jason A. Donenfeld Link: https://patch.msgid.link/20241117212030.629159-2-Jason@zx2c4.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

00d066a4 - netdev_features: convert NETIF_F_LLTX to dev->lltx

Thu, 29 Aug 2024 12:33:37 +0000

netdev_features: convert NETIF_F_LLTX to dev->lltxNETIF_F_LLTX can't be changed via Ethtool and is not a feature,rather an attribute, very similar to IFF_NO_QUEUE (and hot).Free one netdev_features_t bit and make it a "hot" private flag.Signed-off-by: Alexander Lobakin Signed-off-by: Paolo Abeni List of files: /linux-6.15/drivers/net/wireguard/device.c

df9bbb5e - wireguard: device: remove generic .ndo_get_stats64

Thu, 14 Mar 2024 22:49:08 +0000

wireguard: device: remove generic .ndo_get_stats64Commit 3e2f544dd8a33 ("net: get stats64 if device if driver isconfigured") moved the callback to dev_get_tstats64() to net core, so,unless the driver is doing some custom stats collection, it does notneed to set .ndo_get_stats64.Since this driver is now relying in NETDEV_PCPU_STAT_TSTATS, then, itdoesn't need to set the dev_get_tstats64() generic .ndo_get_stats64function pointer.Signed-off-by: Breno Leitao Reviewed-by: Simon Horman Signed-off-by: Jason A. Donenfeld Reviewed-by: Jiri Pirko Signed-off-by: Paolo Abeni List of files: /linux-6.15/drivers/net/wireguard/device.c

db2952df - wireguard: device: leverage core stats allocator

Thu, 14 Mar 2024 22:49:07 +0000

wireguard: device: leverage core stats allocatorWith commit 34d21de99cea9 ("net: Move {l,t,d}stats allocation to coreand convert veth & vrf"), stats allocation could be done on net coreinstead of in this driver.With this new approach, the driver doesn't have to bother with errorhandling (allocation failure checking, making sure free happens in theright spot, etc). This is core responsibility now.Remove the allocation in this driver and leverage the network coreallocation instead.Signed-off-by: Breno Leitao Reviewed-by: Simon Horman Signed-off-by: Jason A. Donenfeld Reviewed-by: Jiri Pirko Signed-off-by: Paolo Abeni List of files: /linux-6.15/drivers/net/wireguard/device.c

93da8d75 - wireguard: use DEV_STATS_INC()

Fri, 17 Nov 2023 14:17:33 +0000

wireguard: use DEV_STATS_INC()wg_xmit() can be called concurrently, KCSAN reported [1]some device stats updates can be lost.Use DEV_STATS_INC() for this unlikely case.[1]BUG: KCSAN: data-race in wg_xmit / wg_xmitread-write to 0xffff888104239160 of 8 bytes by task 1375 on cpu 0:wg_xmit+0x60f/0x680 drivers/net/wireguard/device.c:231__netdev_start_xmit include/linux/netdevice.h:4918 [inline]netdev_start_xmit include/linux/netdevice.h:4932 [inline]xmit_one net/core/dev.c:3543 [inline]dev_hard_start_xmit+0x11b/0x3f0 net/core/dev.c:3559...read-write to 0xffff888104239160 of 8 bytes by task 1378 on cpu 1:wg_xmit+0x60f/0x680 drivers/net/wireguard/device.c:231__netdev_start_xmit include/linux/netdevice.h:4918 [inline]netdev_start_xmit include/linux/netdevice.h:4932 [inline]xmit_one net/core/dev.c:3543 [inline]dev_hard_start_xmit+0x11b/0x3f0 net/core/dev.c:3559...v2: also change wg_packet_consume_data_done() (Hangbin Liu) and wg_packet_purge_staged_packets()Fixes: e7096c131e51 ("net: WireGuard secure network tunnel")Reported-by: syzbot Signed-off-by: Eric Dumazet Cc: Jason A. Donenfeld Cc: Hangbin Liu Signed-off-by: Jason A. Donenfeld Reviewed-by: Hangbin Liu Signed-off-by: David S. Miller List of files: /linux-6.15/drivers/net/wireguard/device.c

d457a0e3 - net: move gso declarations and functions to their own files

Thu, 08 Jun 2023 19:17:37 +0000

net: move gso declarations and functions to their own filesMove declarations into include/net/gso.h and code into net/core/gso.cSigned-off-by: Eric Dumazet Cc: Stanislav Fomichev Reviewed-by: Simon Horman Reviewed-by: David Ahern Link: https://lore.kernel.org/r/20230608191738.3947077-1-edumazet@google.comSigned-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

261e224d - pm/sleep: Add PM_USERSPACE_AUTOSLEEP Kconfig

Thu, 30 Jun 2022 19:12:29 +0000

pm/sleep: Add PM_USERSPACE_AUTOSLEEP KconfigSystems that initiate frequent suspend/resume from userspacecan make the kernel aware by enabling PM_USERSPACE_AUTOSLEEPconfig.This allows for certain sleep-sensitive code (wireguard/rng) todecide on what preparatory work should be performed (or not) intheir pm_notification callbacks.This patch was prompted by the discussion at [1] which attemptsto remove CONFIG_ANDROID that currently guards these code paths.[1] https://lore.kernel.org/r/20220629150102.1582425-1-hch@lst.de/Suggested-by: Jason A. Donenfeld Acked-by: Jason A. Donenfeld Signed-off-by: Kalesh Singh Link: https://lore.kernel.org/r/20220630191230.235306-1-kaleshsingh@google.comSigned-off-by: Greg Kroah-Hartman List of files: /linux-6.15/drivers/net/wireguard/device.c

45ac774c - wireguard: device: check for metadata_dst with skb_valid_dst()

Thu, 21 Apr 2022 13:48:05 +0000

wireguard: device: check for metadata_dst with skb_valid_dst()When we try to transmit an skb with md_dst attached through wireguardwe hit a null pointer dereference in wg_xmit() due to the use ofdst_mtu() which calls into dst_blackhole_mtu() which in turn tries todereference dst->dev.Since wireguard doesn't use md_dsts we should use skb_valid_dst(), whichchecks for DST_METADATA flag, and if it's set, then falls back towireguard's device mtu. That gives us the best chance of transmittingthe packet; otherwise if the blackhole netdev is used we'd getETH_MIN_MTU. [ 263.693506] BUG: kernel NULL pointer dereference, address: 00000000000000e0 [ 263.693908] #PF: supervisor read access in kernel mode [ 263.694174] #PF: error_code(0x0000) - not-present page [ 263.694424] PGD 0 P4D 0 [ 263.694653] Oops: 0000 [#1] PREEMPT SMP NOPTI [ 263.694876] CPU: 5 PID: 951 Comm: mausezahn Kdump: loaded Not tainted 5.18.0-rc1+ #522 [ 263.695190] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014 [ 263.695529] RIP: 0010:dst_blackhole_mtu+0x17/0x20 [ 263.695770] Code: 00 00 00 0f 1f 44 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 10 48 83 e0 fc 8b 40 04 85 c0 75 09 48 8b 07 <8b> 80 e0 00 00 00 c3 66 90 0f 1f 44 00 00 48 89 d7 be 01 00 00 00 [ 263.696339] RSP: 0018:ffffa4a4422fbb28 EFLAGS: 00010246 [ 263.696600] RAX: 0000000000000000 RBX: ffff8ac9c3553000 RCX: 0000000000000000 [ 263.696891] RDX: 0000000000000401 RSI: 00000000fffffe01 RDI: ffffc4a43fb48900 [ 263.697178] RBP: ffffa4a4422fbb90 R08: ffffffff9622635e R09: 0000000000000002 [ 263.697469] R10: ffffffff9b69a6c0 R11: ffffa4a4422fbd0c R12: ffff8ac9d18b1a00 [ 263.697766] R13: ffff8ac9d0ce1840 R14: ffff8ac9d18b1a00 R15: ffff8ac9c3553000 [ 263.698054] FS: 00007f3704c337c0(0000) GS:ffff8acaebf40000(0000) knlGS:0000000000000000 [ 263.698470] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 263.698826] CR2: 00000000000000e0 CR3: 0000000117a5c000 CR4: 00000000000006e0 [ 263.699214] Call Trace: [ 263.699505] [ 263.699759] wg_xmit+0x411/0x450 [ 263.700059] ? bpf_skb_set_tunnel_key+0x46/0x2d0 [ 263.700382] ? dev_queue_xmit_nit+0x31/0x2b0 [ 263.700719] dev_hard_start_xmit+0xd9/0x220 [ 263.701047] __dev_queue_xmit+0x8b9/0xd30 [ 263.701344] __bpf_redirect+0x1a4/0x380 [ 263.701664] __dev_queue_xmit+0x83b/0xd30 [ 263.701961] ? packet_parse_headers+0xb4/0xf0 [ 263.702275] packet_sendmsg+0x9a8/0x16a0 [ 263.702596] ? _raw_spin_unlock_irqrestore+0x23/0x40 [ 263.702933] sock_sendmsg+0x5e/0x60 [ 263.703239] __sys_sendto+0xf0/0x160 [ 263.703549] __x64_sys_sendto+0x20/0x30 [ 263.703853] do_syscall_64+0x3b/0x90 [ 263.704162] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 263.704494] RIP: 0033:0x7f3704d50506 [ 263.704789] Code: 48 c7 c0 ff ff ff ff eb b7 66 2e 0f 1f 84 00 00 00 00 00 90 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 11 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 72 c3 90 55 48 83 ec 30 44 89 4c 24 2c 4c 89 [ 263.705652] RSP: 002b:00007ffe954b0b88 EFLAGS: 00000246 ORIG_RAX: 000000000000002c [ 263.706141] RAX: ffffffffffffffda RBX: 0000558bb259b490 RCX: 00007f3704d50506 [ 263.706544] RDX: 000000000000004a RSI: 0000558bb259b7b2 RDI: 0000000000000003 [ 263.706952] RBP: 0000000000000000 R08: 00007ffe954b0b90 R09: 0000000000000014 [ 263.707339] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffe954b0b90 [ 263.707735] R13: 000000000000004a R14: 0000558bb259b7b2 R15: 0000000000000001 [ 263.708132] [ 263.708398] Modules linked in: bridge netconsole bonding [last unloaded: bridge] [ 263.708942] CR2: 00000000000000e0Fixes: e7096c131e51 ("net: WireGuard secure network tunnel")Link: https://github.com/cilium/cilium/issues/19428Reported-by: Martynas Pumputis Signed-off-by: Nikolay Aleksandrov Acked-by: Daniel Borkmann Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

2d6919c3 - wireguard: device: clear keys on VM fork

Tue, 01 Mar 2022 22:26:55 +0000

wireguard: device: clear keys on VM forkWhen a virtual machine forks, it's important that WireGuard clearexisting sessions so that different plaintexts are not transmitted usingthe same key+nonce, which can result in catastrophic cryptographicfailure. To accomplish this, we simply hook into the newly added vmforknotifier.As a bonus, it turns out that, like the vmfork registration function,the PM registration function is stubbed out when CONFIG_PM_SLEEP is notset, so we can actually just remove the maze of ifdefs, which makes itreally quite clean to support both notifiers at once.Cc: Dominik Brodowski Cc: Greg Kroah-Hartman Cc: Theodore Ts'o Acked-by: Jakub Kicinski Signed-off-by: Jason A. Donenfeld List of files: /linux-6.15/drivers/net/wireguard/device.c

886fcee9 - wireguard: receive: use ring buffer for incoming handshakes

Mon, 29 Nov 2021 15:39:26 +0000

wireguard: receive: use ring buffer for incoming handshakesApparently the spinlock on incoming_handshake's skb_queue is highlycontended, and a torrent of handshake or cookie packets can bring thedata plane to its knees, simply by virtue of enqueueing the handshakepackets to be processed asynchronously. So, we try switching this to aring buffer to hopefully have less lock contention. This alleviates theproblem somewhat, though it still isn't perfect, so future patches willhave to improve this further. However, it at least doesn't completelydiminish the data plane.Reported-by: Streun Fabio Reported-by: Joel Wanner Fixes: e7096c131e51 ("net: WireGuard secure network tunnel")Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

20ae1d6a - wireguard: device: reset peer src endpoint when netns exits

Mon, 29 Nov 2021 15:39:25 +0000

wireguard: device: reset peer src endpoint when netns exitsEach peer's endpoint contains a dst_cache entry that takes a referenceto another netdev. When the containing namespace exits, we take down thesocket and prevent future sockets from being created (by settingcreating_net to NULL), which removes that potential reference on thenetns. However, it doesn't release references to the netns that a netdevcached in dst_cache might be taking, so the netns still might fail toexit. Since the socket is gimped anyway, we can simply clear all thedst_caches (by way of clearing the endpoint src), which will release allreferences.However, the current dst_cache_reset function only releases thosereferences lazily. But it turns out that all of our usages ofwg_socket_clear_peer_endpoint_src are called from contexts that are notexactly high-speed or bottle-necked. For example, when there'sconnection difficulty, or when userspace is reconfiguring the interface.And in particular for this patch, when the netns is exiting. So forthose cases, it makes more sense to call dst_release immediately. Forthat, we add a small helper function to dst_cache.This patch also adds a test to netns.sh from Hangbin Liu to ensure thisdoesn't regress.Tested-by: Hangbin Liu Reported-by: Xiumei Mu Cc: Toke Høiland-Jørgensen Cc: Paolo Abeni Fixes: 900575aa33a3 ("wireguard: device: avoid circular netns references")Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

8b5553ac - wireguard: queueing: get rid of per-peer ring buffers

Mon, 22 Feb 2021 16:25:48 +0000

wireguard: queueing: get rid of per-peer ring buffersHaving two ring buffers per-peer means that every peer results in twomassive ring allocations. On an 8-core x86_64 machine, this commitreduces the per-peer allocation from 18,688 bytes to 1,856 bytes, whichis an 90% reduction. Ninety percent! With some single-machinedeployments approaching 500,000 peers, we're talking about a reductionfrom 7 gigs of memory down to 700 megs of memory.In order to get rid of these per-peer allocations, this commit switchesto using a list-based queueing approach. Currently GSO fragments arechained together using the skb->next pointer (the skb_list_* singlylinked list approach), so we form the per-peer queue around the unusedskb->prev pointer (which sort of makes sense because the links arepointing backwards). Use of skb_queue_* is not possible here, becausethat is based on doubly linked lists and spinlocks. Multiple cores canwrite into the queue at any given time, because its writes occur in thestart_xmit path or in the udp_recv path. But reads happen in a singleworkqueue item per-peer, amounting to a multi-producer, single-consumerparadigm.The MPSC queue is implemented locklessly and never blocks. However, itis not linearizable (though it is serializable), with a very tight andunlikely race on writes, which, when hit (some tiny fraction of the0.15% of partial adds on a fully loaded 16-core x86_64 system), causesthe queue reader to terminate early. However, because every packet sentqueues up the same workqueue item after it is fully added, the workerresumes again, and stopping early isn't actually a problem, since atthat point the packet wouldn't have yet been added to the encryptionqueue. These properties allow us to avoid disabling interrupts orspinning. The design is based on Dmitry Vyukov's algorithm [1].Performance-wise, ordinarily list-based queues aren't preferable toringbuffers, because of cache misses when following pointers around.However, we *already* have to follow the adjacent pointers when workingthrough fragments, so there shouldn't actually be any change there. Apotential downside is that dequeueing is a bit more complicated, but theptr_ring structure used prior had a spinlock when dequeueing, so all andall the difference appears to be a wash.Actually, from profiling, the biggest performance hit, by far, of thiscommit winds up being atomic_add_unless(count, 1, max) and atomic_dec(count), which account for the majority of CPU time, according toperf. In that sense, the previous ring buffer was superior in that itcould check if it was full by head==tail, which the list-based approachcannot do.But all and all, this enables us to get massive memory savings, allowingWireGuard to scale for real world deployments, without taking much of aperformance hit.[1] http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queueReviewed-by: Dmitry Vyukov Reviewed-by: Toke Høiland-Jørgensen Fixes: e7096c131e51 ("net: WireGuard secure network tunnel")Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

99fff526 - wireguard: device: do not generate ICMP for non-IP packets

Mon, 22 Feb 2021 16:25:47 +0000

wireguard: device: do not generate ICMP for non-IP packetsIf skb->protocol doesn't match the actual skb->data header, it'sprobably not a good idea to pass it off to icmp{,v6}_ndo_send, which isexpecting to reply to a valid IP packet. So this commit has that earlymismatch case jump to a later error label.Fixes: e7096c131e51 ("net: WireGuard secure network tunnel")Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

30ac4e2f - wireguard: avoid double unlikely() notation when using IS_ERR()

Mon, 22 Feb 2021 16:25:43 +0000

wireguard: avoid double unlikely() notation when using IS_ERR()The definition of IS_ERR() already applies the unlikely() notationwhen checking the error status of the passed pointer. For thisreason there is no need to have the same notation outside ofIS_ERR() itself.Clean up code by removing redundant notation.Signed-off-by: Antonio Quartulli Signed-off-by: Jason A. Donenfeld Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

42f9e5f0 - wireguard: switch to dev_get_tstats64

Sat, 07 Nov 2020 20:53:19 +0000

wireguard: switch to dev_get_tstats64Replace ip_tunnel_get_stats64() with the new identical core functiondev_get_tstats64().Reviewed-by: Jason A. Donenfeld Signed-off-by: Heiner Kallweit Signed-off-by: Jakub Kicinski List of files: /linux-6.15/drivers/net/wireguard/device.c

01a4967c - wireguard: implement header_ops->parse_protocol for AF_PACKET

Tue, 30 Jun 2020 01:06:20 +0000

wireguard: implement header_ops->parse_protocol for AF_PACKETWireGuard uses skb->protocol to determine packet type, and bails out ifit's not set or set to something it's not expecting. For AF_PACKETinjection, we need to support its call chain of: packet_sendmsg -> packet_snd -> packet_parse_headers -> dev_parse_header_protocol -> parse_protocolWithout a valid parse_protocol, this returns zero, and wireguard thenrejects the skb. So, this wires up the ip_tunnel handler for layer 3packets for that case.Reported-by: Hans Wippel Signed-off-by: Jason A. Donenfeld Signed-off-by: David S. Miller List of files: /linux-6.15/drivers/net/wireguard/device.c