kni: fix freeing order in device release

The "kni_dev" is the private data of the "net_device" in kni, and allocated
with the "net_device" by calling "alloc_netdev()". The "net_device" is
freed by c

kni: fix freeing order in device release

The "kni_dev" is the private data of the "net_device" in kni, and allocated
with the "net_device" by calling "alloc_netdev()". The "net_device" is
freed by calling "free_netdev()" when kni release. The freed memory
includes the "kni_dev". So after "kni_dev" should not be accessed after
"net_device" is released.

Fixes: e77fec694936 ("kni: fix possible mbuf leaks and speed up port release")
Cc: [email protected]

KASAN trace:

[ 85.263717] ==========================================================
[ 85.264418] BUG: KASAN: use-after-free in kni_net_release_fifo_phy+
0x30/0x84 [rte_kni]
[ 85.265139] Read of size 8 at addr ffff000260668d60 by task kni/341
[ 85.265703]
[ 85.265857] CPU: 0 PID: 341 Comm: kni Tainted: G U O
5.15.0-rc4+ #1
[ 85.266525] Hardware name: linux,dummy-virt (DT)
[ 85.266968] Call trace:
[ 85.267220] dump_backtrace+0x0/0x2d0
[ 85.267591] show_stack+0x24/0x30
[ 85.267924] dump_stack_lvl+0x8c/0xb8
[ 85.268294] print_address_description.constprop.0+0x74/0x2b8
[ 85.268855] kasan_report+0x1e4/0x200
[ 85.269224] __asan_load8+0x98/0xd4
[ 85.269577] kni_net_release_fifo_phy+0x30/0x84 [rte_kni]
[ 85.270116] kni_dev_remove.isra.0+0x50/0x64 [rte_kni]
[ 85.270630] kni_ioctl_release+0x254/0x320 [rte_kni]
[ 85.271136] kni_ioctl+0x64/0xb0 [rte_kni]
[ 85.271553] __arm64_sys_ioctl+0xdc/0x120
[ 85.271955] invoke_syscall+0x68/0x1a0
[ 85.272332] el0_svc_common.constprop.0+0x90/0x200
[ 85.272807] do_el0_svc+0x94/0xa4
[ 85.273144] el0_svc+0x78/0x240
[ 85.273463] el0t_64_sync_handler+0x1a8/0x1b0
[ 85.273895] el0t_64_sync+0x1a0/0x1a4
[ 85.274264]
[ 85.274427] Allocated by task 341:
[ 85.274767] kasan_save_stack+0x2c/0x60
[ 85.275157] __kasan_kmalloc+0x90/0xb4
[ 85.275533] __kmalloc_node+0x230/0x594
[ 85.275917] kvmalloc_node+0x8c/0x190
[ 85.276286] alloc_netdev_mqs+0x70/0x6b0
[ 85.276678] kni_ioctl_create+0x224/0xf40 [rte_kni]
[ 85.277166] kni_ioctl+0x9c/0xb0 [rte_kni]
[ 85.277581] __arm64_sys_ioctl+0xdc/0x120
[ 85.277980] invoke_syscall+0x68/0x1a0
[ 85.278357] el0_svc_common.constprop.0+0x90/0x200
[ 85.278830] do_el0_svc+0x94/0xa4
[ 85.279172] el0_svc+0x78/0x240
[ 85.279491] el0t_64_sync_handler+0x1a8/0x1b0
[ 85.279925] el0t_64_sync+0x1a0/0x1a4
[ 85.280292]
[ 85.280454] Freed by task 341:
[ 85.280763] kasan_save_stack+0x2c/0x60
[ 85.281147] kasan_set_track+0x2c/0x40
[ 85.281522] kasan_set_free_info+0x2c/0x50
[ 85.281930] __kasan_slab_free+0xdc/0x140
[ 85.282331] slab_free_freelist_hook+0x90/0x250
[ 85.282782] kfree+0x128/0x580
[ 85.283099] kvfree+0x48/0x60
[ 85.283402] netdev_freemem+0x34/0x44
[ 85.283770] netdev_release+0x50/0x64
[ 85.284138] device_release+0xa0/0x120
[ 85.284516] kobject_put+0xf8/0x160
[ 85.284867] put_device+0x20/0x30
[ 85.285204] free_netdev+0x22c/0x310
[ 85.285562] kni_dev_remove.isra.0+0x48/0x64 [rte_kni]
[ 85.286076] kni_ioctl_release+0x254/0x320 [rte_kni]
[ 85.286573] kni_ioctl+0x64/0xb0 [rte_kni]
[ 85.286992] __arm64_sys_ioctl+0xdc/0x120
[ 85.287392] invoke_syscall+0x68/0x1a0
[ 85.287769] el0_svc_common.constprop.0+0x90/0x200
[ 85.288243] do_el0_svc+0x94/0xa4
[ 85.288579] el0_svc+0x78/0x240
[ 85.288899] el0t_64_sync_handler+0x1a8/0x1b0
[ 85.289332] el0t_64_sync+0x1a0/0x1a4
[ 85.289699]
[ 85.289862] The buggy address belongs to the object at ffff000260668000
[ 85.289862] which belongs to the cache kmalloc-cg-8k of size 8192
[ 85.291079] The buggy address is located 3424 bytes inside of
[ 85.291079] 8192-byte region [ffff000260668000, ffff00026066a000)
[ 85.292213] The buggy address belongs to the page:
[ 85.292684] page:(____ptrval____) refcount:1 mapcount:0 mapping:
0000000000000000 index:0x0 pfn:0x2a0668
[ 85.293585] head:(____ptrval____) order:3 compound_mapcount:0
compound_pincount:0
[ 85.294305] flags: 0xbfff80000010200(slab|head|node=0|zone=2|
lastcpupid=0x7fff)
[ 85.295020] raw: 0bfff80000010200 0000000000000000 dead000000000122
ffff0000c000d680
[ 85.295767] raw: 0000000000000000 0000000080020002 00000001ffffffff
0000000000000000
[ 85.296512] page dumped because: kasan: bad access detected
[ 85.297054]
[ 85.297217] Memory state around the buggy address:
[ 85.297688] ffff000260668c00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb
[ 85.298384] ffff000260668c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb
[ 85.299088] >ffff000260668d00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb
[ 85.299781] ^
[ 85.300396] ffff000260668d80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb
[ 85.301092] ffff000260668e00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb
[ 85.301787] ===========================================================

Signed-off-by: Huisong Li <[email protected]>
Signed-off-by: Min Hu (Connor) <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: fix ioctl signature

Fix kni's ioctl signature to correctly match the kernel's
structs. This shaves off the (void*) casts and uses struct file*
instead of struct inode*. With the correct signatu

kni: fix ioctl signature

Fix kni's ioctl signature to correctly match the kernel's
structs. This shaves off the (void*) casts and uses struct file*
instead of struct inode*. With the correct signature, control flow
integrity checkers are no longer confused at this point.

Signed-off-by: Markus Theil <[email protected]>
Tested-by: Michael Pfeiffer <[email protected]>
Acked-by: Stephen Hemminger <[email protected]>

show more ...

kni: allow configuring thread granularity

The Kni kthreads seem to be re-scheduled at a granularity of roughly
1 millisecond right now, which seems to be insufficient for performing
tests involving

kni: allow configuring thread granularity

The Kni kthreads seem to be re-scheduled at a granularity of roughly
1 millisecond right now, which seems to be insufficient for performing
tests involving a lot of control plane traffic.

Even if KNI_KTHREAD_RESCHEDULE_INTERVAL is set to 5 microseconds, it
seems that the existing code cannot reschedule at the desired granularily,
due to precision constraints of schedule_timeout_interruptible().

In our use case, we leverage the Linux Kernel for control plane, and
it is not uncommon to have 60K - 100K pps for some signaling protocols.

Since we are not in atomic context, the usleep_range() function seems to be
more appropriate for being able to introduce smaller controlled delays,
in the range of 5-10 microseconds. Upon reading the existing code, it would
seem that this was the original intent. Adding sub-millisecond delays,
seems unfeasible with a call to schedule_timeout_interruptible().

KNI_KTHREAD_RESCHEDULE_INTERVAL 5 /* us */
schedule_timeout_interruptible(
usecs_to_jiffies(KNI_KTHREAD_RESCHEDULE_INTERVAL));

Below, we attempted a brief comparison between the existing implementation,
which uses schedule_timeout_interruptible() and usleep_range().

We attempt to measure the CPU usage, and RTT between two Kni interfaces,
which are created on top of vmxnet3 adapters, connected by a vSwitch.

insmod rte_kni.ko kthread_mode=single carrier=on

schedule_timeout_interruptible(usecs_to_jiffies(5))
kni_single CPU Usage: 2-4 %
[root@localhost ~]# ping 1.1.1.2 -I eth1
PING 1.1.1.2 (1.1.1.2) from 1.1.1.1 eth1: 56(84) bytes of data.
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=2.70 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=1.00 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=1.99 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=0.985 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=1.00 ms

usleep_range(5, 10)
kni_single CPU usage: 50%
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.338 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.150 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.123 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=0.139 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=0.159 ms

usleep_range(20, 50)
kni_single CPU usage: 24%
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.202 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.170 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.171 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=0.248 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=0.185 ms

usleep_range(50, 100)
kni_single CPU usage: 13%
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.537 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.257 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.231 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=0.143 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=0.200 ms

usleep_range(100, 200)
kni_single CPU usage: 7%
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.716 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.167 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.459 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=0.455 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=0.252 ms

usleep_range(1000, 1100)
kni_single CPU usage: 2%
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=2.22 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=1.17 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=1.17 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=64 time=1.17 ms
64 bytes from 1.1.1.2: icmp_seq=5 ttl=64 time=1.15 ms

Upon testing, usleep_range(1000, 1100) seems roughly equivalent in
latency and cpu usage to the variant with schedule_timeout_interruptible(),
while usleep_range(100, 200) seems to give a decent tradeoff between
latency and cpu usage, while allowing users to tweak the limits for
improved precision if they have such use cases.

Disabling RTE_KNI_PREEMPT_DEFAULT, interestingly seems to lead to a
softlockup on my kernel.

Kernel panic - not syncing: softlockup: hung tasks
CPU: 0 PID: 1226 Comm: kni_single Tainted: G W O 3.10 #1
<IRQ> [<ffffffff814f84de>] dump_stack+0x19/0x1b
[<ffffffff814f7891>] panic+0xcd/0x1e0
[<ffffffff810993b0>] watchdog_timer_fn+0x160/0x160
[<ffffffff810644b2>] __run_hrtimer.isra.4+0x42/0xd0
[<ffffffff81064b57>] hrtimer_interrupt+0xe7/0x1f0
[<ffffffff8102cd57>] smp_apic_timer_interrupt+0x67/0xa0
[<ffffffff8150321d>] apic_timer_interrupt+0x6d/0x80

This patch also attempts to remove this option.

References:
[1] https://www.kernel.org/doc/Documentation/timers/timers-howto.txt

Signed-off-by: Tudor Cornea <[email protected]>
Acked-by: Padraig Connolly <[email protected]>
Reviewed-by: Ferruh Yigit <[email protected]>

show more ...

kni: restrict bifurcated device support

To enable bifurcated device support, rtnl_lock is released before calling
userspace callbacks and asynchronous requests are enabled.

But these changes caused

kni: restrict bifurcated device support

To enable bifurcated device support, rtnl_lock is released before calling
userspace callbacks and asynchronous requests are enabled.

But these changes caused more issues, like bug #809, #816. To reduce the
scope of the problems, the bifurcated device support related changes are
only enabled when it is requested explicitly with new 'enable_bifurcated'
module parameter.
And bifurcated device support is disabled by default.

So the bifurcated device related problems are isolated and they can be
fixed without impacting all use cases.

Bugzilla ID: 816
Fixes: 631217c76135 ("kni: fix kernel deadlock with bifurcated device")
Cc: [email protected]

Signed-off-by: Ferruh Yigit <[email protected]>
Acked-by: Igor Ryzhov <[email protected]>

show more ...

kni: update kernel API to set random MAC address

Previously used 'random_ether_addr()' API is removed in upstream kernel
with commit
Commit ba530fea8ca1 ("ethernet: remove random_ether_addr()")

Rep

kni: update kernel API to set random MAC address

Previously used 'random_ether_addr()' API is removed in upstream kernel
with commit
Commit ba530fea8ca1 ("ethernet: remove random_ether_addr()")

Replacement API 'eth_random_addr()' is around since v3.6 [1], so
simply switching to this API without any version checks.

[1]
0a4dd594982a ("etherdevice: Rename random_ether_addr to eth_random_addr")

Signed-off-by: Ferruh Yigit <[email protected]>

show more ...

kni: rename variable with namespace prefix

All global variables in kernel should be prefixed by the same
to avoid any symbol conflics. Rename dflt_carrier to kni_default_carrier.

Fixes: 89397a01ce4

kni: rename variable with namespace prefix

All global variables in kernel should be prefixed by the same
to avoid any symbol conflics. Rename dflt_carrier to kni_default_carrier.

Fixes: 89397a01ce4a ("kni: set default carrier state of interface")
Cc: [email protected]

Signed-off-by: Stephen Hemminger <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: support userspace VA

Patch adds support for kernel module to work in IOVA = VA mode by
providing address translation routines to convert userspace VA to
kernel VA.

KNI performance using PA is

kni: support userspace VA

Patch adds support for kernel module to work in IOVA = VA mode by
providing address translation routines to convert userspace VA to
kernel VA.

KNI performance using PA is not changed by this patch.
But comparing KNI using PA to KNI using VA, the latter will have lower
performance due to the cost of the added translation.

This translation is implemented only with kernel versions starting 4.6.0.

Signed-off-by: Vamsi Attunuru <[email protected]>
Signed-off-by: Kiran Kumar K <[email protected]>
Reviewed-by: Jerin Jacob <[email protected]>

show more ...

kni: add ability to set min/max MTU

Starting with kernel version 4.10, there are new min/max MTU values in
net_device structure, which are set to ETH_MIN_MTU and ETH_DATA_LEN by
default. We should b

kni: add ability to set min/max MTU

Starting with kernel version 4.10, there are new min/max MTU values in
net_device structure, which are set to ETH_MIN_MTU and ETH_DATA_LEN by
default. We should be able to change these values to allow MTU more than
1500 to be set on KNI.

Signed-off-by: Igor Ryzhov <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: support minimal ethtool

Some applications use ethtool so add the minimum ethtool ops.

Signed-off-by: Stephen Hemminger <[email protected]>
Acked-by: Ferruh Yigit <[email protected]

kni: support minimal ethtool

Some applications use ethtool so add the minimum ethtool ops.

Signed-off-by: Stephen Hemminger <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: fix copy_from_user failure handling

The correct thing to return if user gives a bad data
is to return -EFAULT. Logging is also discouraged because
it could be used as a DoS attack.

Signed-off-

kni: fix copy_from_user failure handling

The correct thing to return if user gives a bad data
is to return -EFAULT. Logging is also discouraged because
it could be used as a DoS attack.

Signed-off-by: Stephen Hemminger <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: drop unused fields

Several fields were either totally unused or set and never used.

Signed-off-by: Stephen Hemminger <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

kni: remove PCI related information

As there is no ethtool support in KNI anymore,
PCI related information is no longer needed.

Fixes: ea6b39b5b847 ("kni: remove ethtool support")

Signed-off-by: I

kni: remove PCI related information

As there is no ethtool support in KNI anymore,
PCI related information is no longer needed.

Fixes: ea6b39b5b847 ("kni: remove ethtool support")

Signed-off-by: Igor Ryzhov <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: remove ethtool support

Current design requires kernel drivers and they need to be probed by
Linux up to some level so that they can be usable by DPDK for ethtool
support, this requires maintain

kni: remove ethtool support

Current design requires kernel drivers and they need to be probed by
Linux up to some level so that they can be usable by DPDK for ethtool
support, this requires maintaining the Linux drivers in DPDK.

Also ethtool support is limited and hard, if not impossible, to expand
to other PMDs.

Since KNI ethtool support is not used commonly, if not used at all,
removing the support for the sake of simplicity and maintenance.

Signed-off-by: Ferruh Yigit <[email protected]>
Acked-by: Stephen Hemminger <[email protected]>
Acked-by: Neil Horman <[email protected]>
Acked-by: Thomas Monjalon <[email protected]>

show more ...

kni: fix type for MAC address

The type for MAC address should be unsigned.

Fixes: 1cfe212ed17a ("kni: support MAC address change")
Cc: [email protected]

Signed-off-by: Jie Pan <[email protected]>
Acked

kni: fix type for MAC address

The type for MAC address should be unsigned.

Fixes: 1cfe212ed17a ("kni: support MAC address change")
Cc: [email protected]

Signed-off-by: Jie Pan <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>
Reviewed-by: Rami Rosen <[email protected]>

show more ...

eal: remove exec-env directory

Only one header file (rte_kni_common.h) was in the sub-directory
include/exec-env/
This file was installed in a sub-directory of the same name
in the makefile-based b

eal: remove exec-env directory

Only one header file (rte_kni_common.h) was in the sub-directory
include/exec-env/
This file was installed in a sub-directory of the same name
in the makefile-based build.
Source and install directories are moved as below:

lib/librte_eal/linux/eal/include/exec-env/
-> lib/librte_eal/linux/eal/include/

build/include/exec-env/
-> build/include/

The consequence is to have a file hierarchy a bit more flat.

Signed-off-by: Thomas Monjalon <[email protected]>
Reviewed-by: David Marchand <[email protected]>
Tested-by: David Marchand <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: calculate MTU from mbuf size

- mbuf_size and mtu are now being calculated according
to the given mb-pool.

- max_mtu is now being set according to the given mtu

the above two changes provide t

kni: calculate MTU from mbuf size

- mbuf_size and mtu are now being calculated according
to the given mb-pool.

- max_mtu is now being set according to the given mtu

the above two changes provide the ability to work with jumbo frames

Signed-off-by: Liron Himi <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: set default carrier state of interface

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module. The default
carrier state

kni: set default carrier state of interface

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module. The default
carrier state is 'off'.

For KNI interfaces which need to reflect the carrier state of
a physical Ethernet port controlled by the DPDK application, the
default carrier state should be left set to 'off'. The application
can set the carrier state of the KNI interface to reflect the state
of the physical Ethernet port using rte_kni_update_link().

For KNI interfaces which are purely virtual, the default carrier
state can be set to 'on'. This enables the KNI interface to be
used without having to explicity set the carrier state to 'on'
using rte_kni_update_link().

Signed-off-by: Dan Gora <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

kni: fix possible mbuf leaks and speed up port release

rx_q fifo can only be released by kernel thread. There may be
mbuf leaks in rx_q because kernel threads are randomly stopped.

When the kni is

kni: fix possible mbuf leaks and speed up port release

rx_q fifo can only be released by kernel thread. There may be
mbuf leaks in rx_q because kernel threads are randomly stopped.

When the kni is released and netdev is unregisterd, convert the
physical address mbufs in rx_q to the virtual address in free_q.
By the way, alloc_q can be processed together to speed up the
release rate in userspace.

In my test, it is improved from 300-500ms with a mempool that has
131072 mbufs to 10ms(regardless of the specifications).

Suggested-by: Ferruh Yigit <[email protected]>
Signed-off-by: Yangchao Zhou <[email protected]>
Acked-by: Ferruh Yigit <[email protected]>

show more ...

move kernel modules directories

This patch moves the kernel modules code from EAL to a common place.
- Separate the kernel module code from user space code.

Signed-off-by: Hemant Agrawal <hemant.a

move kernel modules directories

This patch moves the kernel modules code from EAL to a common place.
- Separate the kernel module code from user space code.

Signed-off-by: Hemant Agrawal <[email protected]>
Tested-by: Bruce Richardson <[email protected]>

show more ...