LinuxKPI: Add cancel_work() function.

Cancel a work not waiting for it to finish.

Sponsored by: Serenity Cyber Security, LLC
Reviewed by: manu, kib
MFC after: 1 week
Differential Revision: https://

LinuxKPI: Add cancel_work() function.

Cancel a work not waiting for it to finish.

Sponsored by: Serenity Cyber Security, LLC
Reviewed by: manu, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D42811

(cherry picked from commit 1b2f43a7427ebf51561867f6c497833268014512)

show more ...

linuxkpi linux_work: use 'true' instead of 'non-zero'

(cherry picked from commit 96cb1d70008e6d42750e95135316cafd6899384f)

linuxkpi: races between linux_queue_delayed_work_on() and linux_cancel_delayed_work_sync()

(cherry picked from commit 05fe82455f263ad107a860ce20dd89e1a5c1619c)

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

linux(4): Fix a typo in a source code comment

- s/accross/across/

MFC after: 3 days

Fix race condition in linuxkpi workqueue

Consider the following scenario:

1. A delayed_work struct in the WORK_ST_TIMER state.
2. Thread A calls mod_delayed_work()
3. Thread B (a callout thread) si

Fix race condition in linuxkpi workqueue

Consider the following scenario:

1. A delayed_work struct in the WORK_ST_TIMER state.
2. Thread A calls mod_delayed_work()
3. Thread B (a callout thread) simultaneously calls
linux_delayed_work_timer_fn()

The following sequence of events is possible:

A: Call linux_cancel_delayed_work()
A: Change state from TIMER TO CANCEL
B: Change state from CANCEL to TASK
B: taskqueue_enqueue() the task
A: taskqueue_cancel() the task
A: Call linux_queue_delayed_work_on(). This is a no-op because the
state is WORK_ST_TASK.

As a result, the delayed_work struct will never be invoked. This is
causing address resolution in ib_addr.c to stop permanently, as it
never tries to reschedule a task that it thinks is already scheduled.

Fix this by introducing locking into the cancel path (which
corresponds with the lock held while the callout runs). This will
prevent the callout from changing the state of the task until the
cancel is complete, preventing the race.

Differential Revision: https://reviews.freebsd.org/D28420
Reviewed by: hselasky
MFC after: 2 months

show more ...

LinuxKPI: Reimplement irq_work queue on top of fast taskqueue

Summary:
Linux's irq_work queue was created for asynchronous execution of code from contexts where spin_lock's are not available like "h

LinuxKPI: Reimplement irq_work queue on top of fast taskqueue

Summary:
Linux's irq_work queue was created for asynchronous execution of code from contexts where spin_lock's are not available like "hardware interrupt context". FreeBSD's fast taskqueues was created for the same purposes.

Drm-kmod 5.4 uses irq_work_queue() at least in one place to schedule execution of task/work from the critical section that triggers following INVARIANTS-induced panic:

```
panic: acquiring blockable sleep lock with spinlock or critical section held (sleep mutex) linuxkpi_short_wq @ /usr/src/sys/kern/subr_taskqueue.c:281
cpuid = 6
time = 1605048416
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe006b538c90
vpanic() at vpanic+0x182/frame 0xfffffe006b538ce0
panic() at panic+0x43/frame 0xfffffe006b538d40
witness_checkorder() at witness_checkorder+0xf3e/frame 0xfffffe006b538f00
__mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe006b538f50
taskqueue_enqueue() at taskqueue_enqueue+0x42/frame 0xfffffe006b538f70
linux_queue_work_on() at linux_queue_work_on+0xe9/frame 0xfffffe006b538fb0
irq_work_queue() at irq_work_queue+0x21/frame 0xfffffe006b538fd0
semaphore_notify() at semaphore_notify+0xb2/frame 0xfffffe006b539020
__i915_sw_fence_notify() at __i915_sw_fence_notify+0x2e/frame 0xfffffe006b539050
__i915_sw_fence_complete() at __i915_sw_fence_complete+0x63/frame 0xfffffe006b539080
i915_sw_fence_complete() at i915_sw_fence_complete+0x8e/frame 0xfffffe006b5390c0
dma_i915_sw_fence_wake() at dma_i915_sw_fence_wake+0x4f/frame 0xfffffe006b539100
dma_fence_signal_locked() at dma_fence_signal_locked+0x105/frame 0xfffffe006b539180
dma_fence_signal() at dma_fence_signal+0x72/frame 0xfffffe006b5391c0
dma_fence_is_signaled() at dma_fence_is_signaled+0x80/frame 0xfffffe006b539200
dma_resv_add_shared_fence() at dma_resv_add_shared_fence+0xb3/frame 0xfffffe006b539270
i915_vma_move_to_active() at i915_vma_move_to_active+0x18a/frame 0xfffffe006b5392b0
eb_move_to_gpu() at eb_move_to_gpu+0x3ad/frame 0xfffffe006b539320
eb_submit() at eb_submit+0x15/frame 0xfffffe006b539350
i915_gem_do_execbuffer() at i915_gem_do_execbuffer+0x7d4/frame 0xfffffe006b539570
i915_gem_execbuffer2_ioctl() at i915_gem_execbuffer2_ioctl+0x1c1/frame 0xfffffe006b539600
drm_ioctl_kernel() at drm_ioctl_kernel+0xd9/frame 0xfffffe006b539670
drm_ioctl() at drm_ioctl+0x5cd/frame 0xfffffe006b539820
linux_file_ioctl() at linux_file_ioctl+0x323/frame 0xfffffe006b539880
kern_ioctl() at kern_ioctl+0x1f4/frame 0xfffffe006b5398f0
sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe006b5399c0
amd64_syscall() at amd64_syscall+0x121/frame 0xfffffe006b539af0
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe006b539af0
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x800a6f09a, rsp = 0x7fffffffe588, rbp = 0x7fffffffe640 ---
KDB: enter: panic
```
Here, the dma_resv_add_shared_fence() performs a critical_enter() and following call of schedule_work() from semaphore_notify() triggers 'acquiring blockable sleep lock with spinlock or critical section held' panic.

Switching irq_work implementation to fast taskqueue fixes the panic for me.

Other report with the similar bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247166

Reviewed By: hselasky
Differential Revision: https://reviews.freebsd.org/D27171

show more ...

linuxkpi: Add rcu_work functions

The rcu_work function helps to queue some work after waiting for a grace
period.
This is needed by DRM drivers.

Sponsored-by: The FreeBSD Foundation
Reviewed by: hs

linuxkpi: Add rcu_work functions

The rcu_work function helps to queue some work after waiting for a grace
period.
This is needed by DRM drivers.

Sponsored-by: The FreeBSD Foundation
Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D24942

show more ...

Use true and false when dealing with bool type in the LinuxKPI.
No functional change.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Fix synchronous work drain issue in the LinuxKPI.

A work callback may restart itself. Loop in the drain function to see if the
work has been rescheduled and stop the subsequent reschedules, if any.

Fix synchronous work drain issue in the LinuxKPI.

A work callback may restart itself. Loop in the drain function to see if the
work has been rescheduled and stop the subsequent reschedules, if any.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Implement current_work() function in the LinuxKPI.

Tested by: Johannes Lundberg <[email protected]>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Declare and set the global "system_highpri_wq" workqueue structure pointer
in the LinuxKPI.

Submitted by: Johannes Lundberg <[email protected]>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Declare and set the global "system_highpri_wq" workqueue structure pointer
in the LinuxKPI.

Submitted by: Johannes Lundberg <[email protected]>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks

show more ...

Correct the return value from flush_work() and flush_delayed_work() in the
LinuxKPI to comply more with Linux. This fixes an issue when these functions
are used in waiting loops.

MFC after: 1 week
S

Correct the return value from flush_work() and flush_delayed_work() in the
LinuxKPI to comply more with Linux. This fixes an issue when these functions
are used in waiting loops.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Make sure the timer belonging to the delayed work in the LinuxKPI
gets drained before invoking the work function. Else the timer
mutex may still be in use which can lead to use-after-free situations,

Make sure the timer belonging to the delayed work in the LinuxKPI
gets drained before invoking the work function. Else the timer
mutex may still be in use which can lead to use-after-free situations,
because the work function might free the work structure before returning.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Move some other SI_SUB_INIT_IF initializations to SI_SUB_TASKQ

Drop the EARLY_AP_STARTUP gtaskqueue code, as gtaskqueues are now
initialized before APs are started.

Reviewed by: hselasky@, jhb@
Spo

Move some other SI_SUB_INIT_IF initializations to SI_SUB_TASKQ

Drop the EARLY_AP_STARTUP gtaskqueue code, as gtaskqueues are now
initialized before APs are started.

Reviewed by: hselasky@, jhb@
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12054

show more ...

Fix few issues of LinuxKPI workqueue.

LinuxKPI workqueue wrappers reported "successful" cancellation for works
already completed in normal way. This change brings reported status and
real cancellat

Fix few issues of LinuxKPI workqueue.

LinuxKPI workqueue wrappers reported "successful" cancellation for works
already completed in normal way. This change brings reported status and
real cancellation fact into sync. This required for drm-next operation.

Reviewed by: hselasky (earlier version)
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D11904

show more ...

Zero number of CPUs should be translated into the default number of
CPUs when allocating a LinuxKPI workqueue. This also ensures that the
created taskqueue always have a non-zero number of worker thr

Zero number of CPUs should be translated into the default number of
CPUs when allocating a LinuxKPI workqueue. This also ensures that the
created taskqueue always have a non-zero number of worker threads.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Don't create any threads before SI_SUB_INIT_IF in the LinuxKPI. Else
kthread_add() will assert it is called too soon. This fixes a startup
issue when COMPAT_LINUXKPI is in enabled the kernel configur

Don't create any threads before SI_SUB_INIT_IF in the LinuxKPI. Else
kthread_add() will assert it is called too soon. This fixes a startup
issue when COMPAT_LINUXKPI is in enabled the kernel configuration
file.

Reported by: Michael Butler <[email protected]>
MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Fix compilation warning for powerpc64 by not using const keyword in
return types:

Type qualifiers ignored on function return type [-Wreturn-type]

Reported by: andreast @
MFC after: 1 week
Sponsor

Fix compilation warning for powerpc64 by not using const keyword in
return types:

Type qualifiers ignored on function return type [-Wreturn-type]

Reported by: andreast @
MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

LinuxKPI workqueue cleanup.

This change makes the workqueue implementation behave more like in
Linux, both functionality wise and structure wise.

All workqueue code has been moved to linux_work.c

LinuxKPI workqueue cleanup.

This change makes the workqueue implementation behave more like in
Linux, both functionality wise and structure wise.

All workqueue code has been moved to linux_work.c

Add an atomic based statemachine to the work_struct to ensure proper
operation. Prior to this change struct_work was directly mapped to a
FreeBSD task. When a taskqueue has multiple threads the same task may
end up being executed on more than one worker thread simultaneously.
This might cause problems with code coming from Linux, which expects
serial behaviour, similar to Linux tasklets.

Move all global workqueue function names into the linux_xxx domain to
avoid symbol name clashes in the future.

Implement a few more workqueue related functions and macros.

Create two multithreaded taskqueues for the LinuxKPI during module
load, one for time-consuming callbacks and one for non-time consuming
callbacks.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...