rqspinlock: Add locktorture support

Introduce locktorture support for rqspinlock using the newly added
macros as the first in-kernel user and consumer. Guard the code with
CONFIG_BPF_SYSCALL ifdef s

rqspinlock: Add locktorture support

Introduce locktorture support for rqspinlock using the newly added
macros as the first in-kernel user and consumer. Guard the code with
CONFIG_BPF_SYSCALL ifdef since rqspinlock is not available otherwise.

Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

show more ...

torture: Add dowarn argument to torture_sched_setaffinity()

Current use cases of torture_sched_setaffinity() are well served by its
unconditional warning on error. However, an upcoming use case for

torture: Add dowarn argument to torture_sched_setaffinity()

Current use cases of torture_sched_setaffinity() are well served by its
unconditional warning on error. However, an upcoming use case for a
preemption kthread needs to avoid warnings that might otherwise arise
when that kthread attempted to bind itself to a CPU on its way offline.
This commit therefore adds a dowarn argument that, when false, suppresses
the warning.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Uladzislau Rezki (Sony) <[email protected]>

show more ...

locktorture: Add MODULE_DESCRIPTION()

Fix the 'make W=1' warning:
WARNING: modpost: missing MODULE_DESCRIPTION() in kernel/locking/locktorture.o

Signed-off-by: Jeff Johnson <[email protected]

locktorture: Add MODULE_DESCRIPTION()

Fix the 'make W=1' warning:
WARNING: modpost: missing MODULE_DESCRIPTION() in kernel/locking/locktorture.o

Signed-off-by: Jeff Johnson <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Increase Hamming distance between call_rcu_chain and rcu_call_chains

One letter difference is really not enough, so this commit changes
call_rcu_chain to call_rcu_chain_list.

Reported-

locktorture: Increase Hamming distance between call_rcu_chain and rcu_call_chains

One letter difference is really not enough, so this commit changes
call_rcu_chain to call_rcu_chain_list.

Reported-by: Dan Carpenter <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Neeraj Upadhyay (AMD) <[email protected]>

show more ...

locktorture: Check the correct variable for allocation failure

There is a typo so this checks the wrong variable. "chains" plural vs
"chain" singular. We already know that "chains" is non-zero.

F

locktorture: Check the correct variable for allocation failure

There is a typo so this checks the wrong variable. "chains" plural vs
"chain" singular. We already know that "chains" is non-zero.

Fixes: 7f993623e9eb ("locktorture: Add call_rcu_chains module parameter")
Signed-off-by: Dan Carpenter <[email protected]>
Reviewed-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Rename readers_bind/writers_bind to bind_readers/bind_writers

This commit renames the readers_bind and writers_bind module parameters
to bind_readers and bind_writers, respectively. Th

locktorture: Rename readers_bind/writers_bind to bind_readers/bind_writers

This commit renames the readers_bind and writers_bind module parameters
to bind_readers and bind_writers, respectively. This provides added
clarity via the imperative mode and better organizes the documentation.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Add call_rcu_chains module parameter

When running locktorture on large systems, there will normally be
enough RCU activity to ensure that there is a grace period in flight
at all times.

locktorture: Add call_rcu_chains module parameter

When running locktorture on large systems, there will normally be
enough RCU activity to ensure that there is a grace period in flight
at all times. However, on smaller systems, RCU might well be idle the
majority of the time. This situation can be inconvenient in cases where
the RCU CPU stall warning is part of the debugging process.

This commit therefore adds an call_rcu_chains module parameter to
locktorture, allowing the user to specify the desired number of
self-propagating call_rcu() chains. For good measure, immediately
before invoking call_rcu(), the self-propagating RCU callback invokes
start_poll_synchronize_rcu() to force the immediate start of a grace
period, with the call_rcu() forcing another to start shortly thereafter.

Booting with locktorture.call_rcu_chains=2 increases the probability
of a stuck locking primitive resulting in an RCU CPU stall warning from
about 25% to nearly 100%.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Add new module parameters to lock_torture_print_module_parms()

This commit adds new module parameters to lock_torture_print_module_parms,
and alphabetizes things while in the area. Thi

locktorture: Add new module parameters to lock_torture_print_module_parms()

This commit adds new module parameters to lock_torture_print_module_parms,
and alphabetizes things while in the area. This change makes locktorture
test results more useful and self-contained.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Add acq_writer_lim to complain about long acquistion times

This commit adds a locktorture.acq_writer_lim module parameter that
specifies the maximum number of jiffies that is expected t

locktorture: Add acq_writer_lim to complain about long acquistion times

This commit adds a locktorture.acq_writer_lim module parameter that
specifies the maximum number of jiffies that is expected to be consumed
by write-side lock acquisition. If this limit is exceeded, a WARN_ONCE()
causes a splat. Note that this limit applies to the main lock acquisition
only, not to any nested acquisitions.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Consolidate "if" statements in lock_torture_writer()

There is a pair of adjacent "if" statements with identical conditions in
the lock_torture_writer() function. This commit therefore

locktorture: Consolidate "if" statements in lock_torture_writer()

There is a pair of adjacent "if" statements with identical conditions in
the lock_torture_writer() function. This commit therefore combines them.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Alphabetize torture_param() entries

There are getting to be too many module parameters for a random list to be
comfortable, so this commit alphabetizes the list. Strictly code motion.

locktorture: Alphabetize torture_param() entries

There are getting to be too many module parameters for a random list to be
comfortable, so this commit alphabetizes the list. Strictly code motion.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

locktorture: Add readers_bind and writers_bind module parameters

This commit adds readers_bind and writers_bind module parameters to
locktorture in order to skew tests across socket boundaries. Thi

locktorture: Add readers_bind and writers_bind module parameters

This commit adds readers_bind and writers_bind module parameters to
locktorture in order to skew tests across socket boundaries. This skewing
is intended to provide additional variable-latency stress on the primitive
under test.

Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Frederic Weisbecker <[email protected]>

show more ...

torture: Add lock_torture writer_fifo module parameter

This commit adds a module parameter that causes the locktorture writer
to run at real-time priority.

To use it:
insmod /lib/modules/torture.ko

torture: Add lock_torture writer_fifo module parameter

This commit adds a module parameter that causes the locktorture writer
to run at real-time priority.

To use it:
insmod /lib/modules/torture.ko random_shuffle=1
insmod /lib/modules/locktorture.ko torture_type=mutex_lock rt_boost=1 rt_boost_factor=50 nested_locks=3 writer_fifo=1
^^^^^^^^^^^^^

A predecessor to this patch has been helpful to uncover issues with the
proxy-execution series.

[ paulmck: Remove locktorture-specific code from kernel/torture.c. ]

Cc: "Paul E. McKenney" <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: [email protected]
Signed-off-by: Dietmar Eggemann <[email protected]>
[jstultz: Include header change to build, reword commit message]
Signed-off-by: John Stultz <[email protected]>
Acked-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Add long_hold to adjust lock-hold delays

This commit adds a long_hold module parameter to allow testing diagnostics
for excessive lock-hold times. Also adjust torture_param() invocatio

locktorture: Add long_hold to adjust lock-hold delays

This commit adds a long_hold module parameter to allow testing diagnostics
for excessive lock-hold times. Also adjust torture_param() invocations
for longer line length while in the area.

Signed-off-by: Paul E. McKenney <[email protected]>
Reviewed-by: Joel Fernandes (Google) <[email protected]>

show more ...

locktorture: Add raw_spinlock* torture tests for PREEMPT_RT kernels

In PREEMPT_RT kernels, both spin_lock() and spin_lock_irq() are converted
to sleepable rt_spin_lock(). This means that the interr

locktorture: Add raw_spinlock* torture tests for PREEMPT_RT kernels

In PREEMPT_RT kernels, both spin_lock() and spin_lock_irq() are converted
to sleepable rt_spin_lock(). This means that the interrupt related
suffixes for spin_lock/unlock(_irq, irqsave/irqrestore) do not affect
the CPU's interrupt state. This commit therefore adds raw spin-lock
torture tests. This in turn permits pure spin locks to be tested in
PREEMPT_RT kernels.

Suggested-by: Paul E. McKenney <[email protected]>
Suggested-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Zqiang <[email protected]>
Reviewed-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: With nested locks, occasionally skip main lock

If we're using nested locking to stress things, occasionally
skip taking the main lock, so that we can get some different
contention patte

locktorture: With nested locks, occasionally skip main lock

If we're using nested locking to stress things, occasionally
skip taking the main lock, so that we can get some different
contention patterns between the writers (to hopefully get two
disjoint blocked trees)

Cc: Davidlohr Bueso <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: [email protected]
Co-developed-by: Connor O'Brien <[email protected]>
Signed-off-by: Connor O'Brien <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Add nested locking to rtmutex torture tests

This patch adds randomized nested locking to the rtmutex torture
tests. Additionally it adds LOCK09 config files for testing
rtmutexes with n

locktorture: Add nested locking to rtmutex torture tests

This patch adds randomized nested locking to the rtmutex torture
tests. Additionally it adds LOCK09 config files for testing
rtmutexes with nested locking.

Cc: Davidlohr Bueso <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: [email protected]
Reviewed-by: Davidlohr Bueso <[email protected]>
Co-developed-by: Connor O'Brien <[email protected]>
Signed-off-by: Connor O'Brien <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Add nested locking to mutex torture tests

This patch adds randomized nested locking to the mutex torture
tests, as well as new LOCK08 config files for testing mutexes
with nested lockin

locktorture: Add nested locking to mutex torture tests

This patch adds randomized nested locking to the mutex torture
tests, as well as new LOCK08 config files for testing mutexes
with nested locking

Cc: Davidlohr Bueso <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: [email protected]
Reviewed-by: Davidlohr Bueso <[email protected]>
Co-developed-by: Connor O'Brien <[email protected]>
Signed-off-by: Connor O'Brien <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Add nested_[un]lock() hooks and nlocks parameter

In order to extend locktorture to support lock nesting, add
nested_lock() and nested_unlock() hooks to the torture ops.

These take a 32

locktorture: Add nested_[un]lock() hooks and nlocks parameter

In order to extend locktorture to support lock nesting, add
nested_lock() and nested_unlock() hooks to the torture ops.

These take a 32bit lockset mask which is generated at random,
so some number of locks will be taken before the main lock is
taken and released afterwards.

Additionally, add nested_locks module parameter to allow
specifying the number of nested locks to be used.

This has been helpful to uncover issues in the proxy-exec
series development.

This was inspired by locktorture extensions originally implemented
by Connor O'Brien, for stress testing the proxy-execution series:
https://lore.kernel.org/lkml/[email protected]/

Cc: Davidlohr Bueso <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: [email protected]
Reviewed-by: Davidlohr Bueso <[email protected]>
Co-developed-by: Connor O'Brien <[email protected]>
Signed-off-by: Connor O'Brien <[email protected]>
Signed-off-by: John Stultz <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Make the rt_boost factor a tunable

The rt boosting in locktorture has a factor variable s currently large enough
that boosting only happens once every minute or so. Add a tunable to red

locktorture: Make the rt_boost factor a tunable

The rt boosting in locktorture has a factor variable s currently large enough
that boosting only happens once every minute or so. Add a tunable to reduce the
factor so that boosting happens more often, to test paths and arrive at failure
modes earlier. With this change, I can set the factor to like 50 and have the
boosting happens every 10 seconds or so.

Tested with boot parameters:
locktorture.torture_type=mutex_lock
locktorture.onoff_interval=1
locktorture.nwriters_stress=8
locktorture.stutter=0
locktorture.rt_boost=1
locktorture.rt_boost_factor=50
locktorture.nlocks=3

Signed-off-by: Joel Fernandes (Google) <[email protected]>
Reviewed-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Allow non-rtmutex lock types to be boosted

Currently RT boosting is only done for rtmutex_lock, however with proxy
execution, we also have the mutex_lock participating in priorities. To

locktorture: Allow non-rtmutex lock types to be boosted

Currently RT boosting is only done for rtmutex_lock, however with proxy
execution, we also have the mutex_lock participating in priorities. To
exercise the testing better, add RT boosting to other lock testing types
as well, using a new knob (rt_boost).

Tested with boot parameters:
locktorture.torture_type=mutex_lock
locktorture.onoff_interval=1
locktorture.nwriters_stress=8
locktorture.stutter=0
locktorture.rt_boost=1
locktorture.rt_boost_factor=1
locktorture.nlocks=3

Signed-off-by: Joel Fernandes (Google) <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture,rcutorture,torture: Always log error message

Unconditionally log messages corresponding to errors.

Acked-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Li Zhijian <zhijianx.li@inte

locktorture,rcutorture,torture: Always log error message

Unconditionally log messages corresponding to errors.

Acked-by: Davidlohr Bueso <[email protected]>
Signed-off-by: Li Zhijian <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Warn on individual lock_torture_init() error conditions

When running locktorture as a module, any lock_torture_init() issues will be
reflected in the error code from modprobe or insmod,

locktorture: Warn on individual lock_torture_init() error conditions

When running locktorture as a module, any lock_torture_init() issues will be
reflected in the error code from modprobe or insmod, as the case may be.
However, these error codes are not available when running locktorture
built-in, for example, when using the kvm.sh script. This commit
therefore adds WARN_ON_ONCE() to allow distinguishing lock_torture_init()
errors when running locktorture built-in.

Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Count lock readers

Currently, the lock_is_read_held variable is bool, so that a reader sets
it to true just after lock acquisition and then to false just before
lock release. This work

locktorture: Count lock readers

Currently, the lock_is_read_held variable is bool, so that a reader sets
it to true just after lock acquisition and then to false just before
lock release. This works in a rough statistical sense, but can result
in false negatives just after one of a pair of concurrent readers has
released the lock. This approach does have low overhead, but at the
expense of the setting to true potentially never leaving the reader's
store buffer, thus resulting in an unconditional false negative.

This commit therefore converts this variable to atomic_t and makes
the reader use atomic_inc() just after acquisition and atomic_dec()
just before release. This does increase overhead, but this increase is
negligible compared to the 10-microsecond lock hold time.

Signed-off-by: Paul E. McKenney <[email protected]>

show more ...

locktorture: Mark statistics data races

The lock_stress_stats structure's ->n_lock_fail and ->n_lock_acquired
fields are incremented and sampled locklessly using plain C-language
statements, which K

locktorture: Mark statistics data races

The lock_stress_stats structure's ->n_lock_fail and ->n_lock_acquired
fields are incremented and sampled locklessly using plain C-language
statements, which KCSAN objects to. This commit therefore marks the
statistics gathering with data_race() to flag the intent. While in
the area, this commit also reduces the number of accesses to the
->n_lock_acquired field, thus eliminating some possible check/use
confusion.

Signed-off-by: Paul E. McKenney <[email protected]>

show more ...