Changes in Kconfig

1dc1e0b9 - srcu: Make FORCE_NEED_SRCU_NMI_SAFE depend on RCU_EXPERT

Tue, 25 Mar 2025 14:31:45 +0000

srcu: Make FORCE_NEED_SRCU_NMI_SAFE depend on RCU_EXPERTThe FORCE_NEED_SRCU_NMI_SAFE is useful only for those wishing to testthe SRCU code paths that accommodate architectures that do not haveNMI-safe per-CPU operations, that is, those architectures that do notselect the ARCH_HAS_NMI_SAFE_THIS_CPU_OPS Kconfig option. As such, thisis a specialized Kconfig option that is not intended for casual users.This commit therefore hides it behind the RCU_EXPERT Kconfig option.Given that this new FORCE_NEED_SRCU_NMI_SAFE Kconfig option has no effectunless the ARCH_HAS_NMI_SAFE_THIS_CPU_OPS Kconfig option is also selected,it also depends on this Kconfig option.[ paulmck: Apply Geert Uytterhoeven feedback. ][ boqun: Add the "Fixes" tag. ]Reported-by: Geert Uytterhoeven Closes: https://lore.kernel.org/all/CAMuHMdX6dy9_tmpLkpcnGzxyRbe6qSWYukcPp=H1GzZdyd3qBQ@mail.gmail.com/Fixes: 536e8b9b80bc ("srcu: Add FORCE_NEED_SRCU_NMI_SAFE Kconfig for testing")Signed-off-by: Paul E. McKenney Reviewed-by: Geert Uytterhoeven Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

8437bb84 - rcu: limit PREEMPT_RCU configurations

Fri, 13 Dec 2024 04:06:58 +0000

rcu: limit PREEMPT_RCU configurationsPREEMPT_LAZY can be enabled stand-alone or alongside PREEMPT_DYNAMICwhich allows for dynamic switching of preemption models.The choice of PREEMPT_RCU or not, however, is fixed at compile time.Given that PREEMPT_RCU makes some trade-offs to optimize for latencyas opposed to throughput, configurations with limited preemptionmight prefer the stronger forward-progress guarantees of PREEMPT_RCU=n.Accordingly, explicitly limit PREEMPT_RCU=y to the latency orientedpreemption models: PREEMPT, PREEMPT_RT, and the runtime configurablemodel PREEMPT_DYNAMIC.This means the throughput oriented models, PREEMPT_NONE,PREEMPT_VOLUNTARY, and PREEMPT_LAZY will run with PREEMPT_RCU=n.Cc: Paul E. McKenney Cc: Peter Zijlstra Reviewed-by: Frederic Weisbecker Signed-off-by: Ankur Arora Signed-off-by: Paul E. McKenney Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

536e8b9b - srcu: Add FORCE_NEED_SRCU_NMI_SAFE Kconfig for testing

Fri, 10 Jan 2025 19:26:23 +0000

srcu: Add FORCE_NEED_SRCU_NMI_SAFE Kconfig for testingThe srcu_read_lock_nmisafe() and srcu_read_unlock_nmisafe() functionsmap to __srcu_read_lock() and __srcu_read_unlock() on systems like x86that have NMI-safe this_cpu_inc() operations. This makes the underlying__srcu_read_lock_nmisafe() and __srcu_read_unlock_nmisafe() functionsdifficult to test on (for example) x86 systems, allowing bugs to creep in.This commit therefore creates a FORCE_NEED_SRCU_NMI_SAFE Kconfig thatforces those underlying functions to be used even on systems where theyare not needed, thus providing better testing coverage.Signed-off-by: Paul E. McKenney Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

4dca1af4 - rcu: rename PREEMPT_AUTO to PREEMPT_LAZY

Fri, 13 Dec 2024 04:06:53 +0000

rcu: rename PREEMPT_AUTO to PREEMPT_LAZYReplace mentions of PREEMPT_AUTO with PREEMPT_LAZY.Also, since PREMPT_LAZY implies PREEMPTION, we can reduce theTASKS_RCU selection criteria from this: NEED_TASKS_RCU && (PREEMPTION || PREEMPT_AUTO)to this: NEED_TASKS_RCU && PREEMPTIONCC: Paul E. McKenney Reviewed-by: Frederic Weisbecker Reviewed-by: Sebastian Andrzej Siewior Signed-off-by: Ankur Arora Signed-off-by: Paul E. McKenney Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

81a208c5 - rcu: Clarify RCU_LAZY and RCU_LAZY_DEFAULT_OFF help text

Thu, 09 Jan 2025 03:50:37 +0000

rcu: Clarify RCU_LAZY and RCU_LAZY_DEFAULT_OFF help textThis commit wordsmiths the RCU_LAZY and RCU_LAZY_DEFAULT_OFF Kconfigoptions' help text.Signed-off-by: Paul E. McKenney Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

f30e2582 - rcu: Add rcuog kthreads to RCU_NOCB_CPU help text

Wed, 09 Oct 2024 18:00:39 +0000

rcu: Add rcuog kthreads to RCU_NOCB_CPU help textThe RCU_NOCB_CPU help text currently fails to mention rcuog kthreads,so this commit adds this information.Reported-by: Olivier Langlois Signed-off-by: Paul E. McKenney Reviewed-by: Neeraj Upadhyay Signed-off-by: Frederic Weisbecker List of files: /linux-6.15/kernel/rcu/Kconfig

1b4e9fdf - rcu: Create NEED_TASKS_RCU to factor out enablement logic

Thu, 22 Feb 2024 18:09:19 +0000

rcu: Create NEED_TASKS_RCU to factor out enablement logicCurrently, if a Kconfig option depends on TASKS_RCU, it conditionally does"select TASKS_RCU if PREEMPTION". This works, but requires any change inthis enablement logic to be replicated across all such "select" clauses.This commit therefore creates a new NEED_TASKS_RCU Kconfig option sothat the default value of TASKS_RCU can depend on a combination of thisnew option and any needed enablement logic, so that this logic is inone place.While in the area, also anticipate a likely future change by addingPREEMPT_AUTO to that logic.Signed-off-by: Paul E. McKenney Cc: Ankur Arora Cc: Thomas Gleixner Cc: Steven Rostedt Acked-by: Mark Rutland Signed-off-by: Uladzislau Rezki (Sony) List of files: /linux-6.15/kernel/rcu/Kconfig

c1ec7c15 - rcu: Make TINY_RCU depend on !PREEMPT_RCU rather than !PREEMPTION

Wed, 14 Feb 2024 23:33:55 +0000

rcu: Make TINY_RCU depend on !PREEMPT_RCU rather than !PREEMPTIONRight now, TINY_RCU depends on (!PREEMPTION && !SMP), which has served thekernel well for many years due to the fact that PREEMPT_RCU is normallya synonym for PREEMPTION. But with the advent of lazy preemption,it will be possible to have non-preemptible RCU in a preemptible kernel,so that kernels could be built with PREEMPT_RCU=n and PREEMPTION=y.This commit therefore makes TINY_RCU depend on (!PREEMPT_RCU && !SMP),thus allowing for a non-preemptible RCU in preemptible kernels.Signed-off-by: Paul E. McKenney Cc: Ankur Arora Cc: Thomas Gleixner Signed-off-by: Uladzislau Rezki (Sony) List of files: /linux-6.15/kernel/rcu/Kconfig

7f66f099 - rcu: Provide a boot time parameter to control lazy RCU

Sun, 03 Dec 2023 01:12:52 +0000

rcu: Provide a boot time parameter to control lazy RCUTo allow more flexible arrangements while still provide a single kernelfor distros, provide a boot time parameter to enable/disable lazy RCU.Specify: rcutree.enable_rcu_lazy=[y|1|n|0]Which also requires rcu_nocbs=allat boot time to enable/disable lazy RCU.To disable it by default at build time when CONFIG_RCU_LAZY=y, the newCONFIG_RCU_LAZY_DEFAULT_OFF can be used.Signed-off-by: Qais Yousef (Google) Tested-by: Andrea Righi Reviewed-by: Paul E. McKenney Signed-off-by: Boqun Feng List of files: /linux-6.15/kernel/rcu/Kconfig

f51164a8 - rcu: Employ jiffies-based backstop to callback time limit

Fri, 31 Mar 2023 16:05:56 +0000

rcu: Employ jiffies-based backstop to callback time limitCurrently, if there are more than 100 ready-to-invoke RCU callbacks queuedon a given CPU, the rcu_do_batch() function sets a timeout for invocationof the series. This timeout defaulting to three milliseconds, and maybe adjusted using the rcutree.rcu_resched_ns kernel boot parameter.This timeout is checked using local_clock(), but the overhead of thisfunction combined with the common-case very small callback-invocationoverhead means that local_clock() is checked every 32nd invocation.This works well except for longer-than average callbacks. For example,a series of 500-microsecond-duration callbacks means that local_clock()is checked only once every 16 milliseconds, which makes it difficult toenforce a three-millisecond timeout.This commit therefore adds a Kconfig option RCU_DOUBLE_CHECK_CB_TIMEthat enables backup timeout checking using the coarser grained butlighter weight jiffies. If the jiffies counter detects a timeout,then local_clock() is consulted even if this is not the 32nd callback.This prevents the aforementioned 16-millisecond latency blow.Reported-by: Domas Mituzas Signed-off-by: Paul E. McKenney List of files: /linux-6.15/kernel/rcu/Kconfig

e035e887 - rcu: Remove CONFIG_SRCU

Fri, 24 Mar 2023 20:52:49 +0000

rcu: Remove CONFIG_SRCUNow that all references to CONFIG_SRCU have been removed, it is time toremove CONFIG_SRCU itself.Signed-off-by: Paul E. McKenney Cc: John Ogness Cc: Petr Mladek Reviewed-by: John Ogness Signed-off-by: Joel Fernandes (Google) List of files: /linux-6.15/kernel/rcu/Kconfig

0cd7e350 - rcu: Make SRCU mandatory

Tue, 22 Nov 2022 21:53:57 +0000

rcu: Make SRCU mandatoryKernels configured with CONFIG_PRINTK=n and CONFIG_SRCU=n get buildfailures. This causes trouble for deep embedded systems. But giventhat there are more than 25 instances of "select SRCU" in the kernel,it is hard to believe that there are many kernels running in productionwithout SRCU. This commit therefore makes SRCU mandatory. The SRCUKconfig option remains for backwards compatibility, and will be removedwhen it is no longer used.[ paulmck: Update per kernel test robot feedback. ]Reported-by: John Ogness Reported-by: Petr Mladek Signed-off-by: Paul E. McKenney Cc: Acked-by: Randy Dunlap # build-testedReviewed-by: John Ogness List of files: /linux-6.15/kernel/rcu/Kconfig

3cb278e7 - rcu: Make call_rcu() lazy to save power

Sun, 16 Oct 2022 16:22:54 +0000

rcu: Make call_rcu() lazy to save powerImplement timer-based RCU callback batching (also known as lazycallbacks). With this we save about 5-10% of power consumed dueto RCU requests that happen when system is lightly loaded or idle.By default, all async callbacks (queued via call_rcu) are markedlazy. An alternate API call_rcu_hurry() is provided for the few users,for example synchronize_rcu(), that need the old behavior.The batch is flushed whenever a certain amount of time has passed, orthe batch on a particular CPU grows too big. Also memory pressure willflush it in a future patch.To handle several corner cases automagically (such as rcu_barrier() andhotplug), we re-use bypass lists which were originally introduced toaddress lock contention, to handle lazy CBs as well. The bypass listlength has the lazy CB length included in it. A separate lazy CB lengthcounter is also introduced to keep track of the number of lazy CBs.[ paulmck: Fix formatting of inline call_rcu_lazy() definition. ][ paulmck: Apply Zqiang feedback. ][ paulmck: Apply s/call_rcu_flush/call_rcu_hurry/ feedback from Tejun Heo. ]Suggested-by: Paul McKenney Acked-by: Frederic Weisbecker Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney List of files: /linux-6.15/kernel/rcu/Kconfig

2e83b879 - srcu: Create an srcu_read_lock_nmisafe() and srcu_read_unlock_nmisafe()

Thu, 15 Sep 2022 21:29:07 +0000

srcu: Create an srcu_read_lock_nmisafe() and srcu_read_unlock_nmisafe()On strict load-store architectures, the use of this_cpu_inc() bysrcu_read_lock() and srcu_read_unlock() is not NMI-safe in TREE SRCU.To see this suppose that an NMI arrives in the middle of srcu_read_lock(),just after it has read ->srcu_lock_count, but before it has writtenthe incremented value back to memory. If that NMI handler also doessrcu_read_lock() and srcu_read_lock() on that same srcu_struct structure,then upon return from that NMI handler, the interrupted srcu_read_lock()will overwrite the NMI handler's update to ->srcu_lock_count, butleave unchanged the NMI handler's update by srcu_read_unlock() to->srcu_unlock_count.This can result in a too-short SRCU grace period, which can in turnresult in arbitrary memory corruption.If the NMI handler instead interrupts the srcu_read_unlock(), thiscan result in eternal SRCU grace periods, which is not much better.This commit therefore creates a pair of new srcu_read_lock_nmisafe()and srcu_read_unlock_nmisafe() functions, which allow SRCU readers inboth NMI handlers and in process and IRQ context. It is bad practiceto mix the existing and the new _nmisafe() primitives on the samesrcu_struct structure. Use one set or the other, not both.Just to underline that "bad practice" point, using srcu_read_lock() atprocess level and srcu_read_lock_nmisafe() in your NMI handler will not,repeat NOT, work. If you do not immediately understand why this is thecase, please review the earlier paragraphs in this commit log.[ paulmck: Apply kernel test robot feedback. ][ paulmck: Apply feedback from Randy Dunlap. ][ paulmck: Apply feedback from John Ogness. ][ paulmck: Apply feedback from Frederic Weisbecker. ]Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by: Paul E. McKenney Acked-by: Randy Dunlap # build-testedReviewed-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: John Ogness Cc: Petr Mladek List of files: /linux-6.15/kernel/rcu/Kconfig

8f489b4d - rcu/nocb: Add option to opt rcuo kthreads out of RT priority

Wed, 11 May 2022 08:57:03 +0000

rcu/nocb: Add option to opt rcuo kthreads out of RT priorityThis commit introduces a RCU_NOCB_CPU_CB_BOOST Kconfig option thatprevents rcuo kthreads from running at real-time priority, even inkernels built with RCU_BOOST. This capability is important to devicesneeding low-latency (as in a few milliseconds) response from expeditedRCU grace periods, but which are not running a classic real-time workload.On such devices, permitting the rcuo kthreads to run at real-time priorityresults in unacceptable latencies imposed on the application tasks,which run as SCHED_OTHER.See for example the following trace output:<...>-60 [006] d..1 2979.028717: rcu_batch_start: rcu_preempt CBs=34619 bl=270If that rcuop kthread were permitted to run at real-time SCHED_FIFOpriority, it would monopolize its CPU for hundreds of millisecondswhile invoking those 34619 RCU callback functions, which would cause anunacceptably long latency spike for many application stacks on Androidplatforms.However, some existing real-time workloads require that callbackinvocation run at SCHED_FIFO priority, for example, those running onsystems with heavy SCHED_OTHER background loads. (It is the real-timesystem's administrator's responsibility to make sure that importantreal-time tasks run at a higher priority than do RCU's kthreads.)Therefore, this new RCU_NOCB_CPU_CB_BOOST Kconfig option defaults to"y" on kernels built with PREEMPT_RT and defaults to "n" otherwise.The effect is to preserve current behavior for real-time systems, but forother systems to allow expedited RCU grace periods to run with real-timepriority while continuing to invoke RCU callbacks as SCHED_OTHER.As you would expect, this RCU_NOCB_CPU_CB_BOOST Kconfig option has noeffect except on CPUs with offloaded RCU callbacks.Signed-off-by: Uladzislau Rezki (Sony) Signed-off-by: Paul E. McKenney Acked-by: Joel Fernandes (Google) Reviewed-by: Neeraj Upadhyay List of files: /linux-6.15/kernel/rcu/Kconfig

b37a667c - rcu/nocb: Add an option to offload all CPUs on boot

Fri, 22 Apr 2022 17:52:47 +0000

rcu/nocb: Add an option to offload all CPUs on bootSystems built with CONFIG_RCU_NOCB_CPU=y but booted without eitherthe rcu_nocbs= or rcu_nohz_full= kernel-boot parameters will not havecallback offloading on any of the CPUs, nor can any of the CPUs beswitched to enable callback offloading at runtime. Although this isintentional, it would be nice to have a way to offload all the CPUswithout having to make random bootloaders specify either the rcu_nocbs=or the rcu_nohz_full= kernel-boot parameters.This commit therefore provides a new CONFIG_RCU_NOCB_CPU_DEFAULT_ALLKconfig option that switches the default so as to offload callbackprocessing on all of the CPUs. This default can still be overriddenusing the rcu_nocbs= and rcu_nohz_full= kernel-boot parameters.Reviewed-by: Kalesh Singh Reviewed-by: Uladzislau Rezki (In v4.1, fixed issues with CONFIG maze reported by kernel test robot).Reported-by: kernel test robot Signed-off-by: Joel Fernandes Signed-off-by: Paul E. McKenney Reviewed-by: Neeraj Upadhyay List of files: /linux-6.15/kernel/rcu/Kconfig

e67198cc - context_tracking: Take idle eqs entrypoints over RCU

Wed, 08 Jun 2022 14:40:25 +0000

context_tracking: Take idle eqs entrypoints over RCUThe RCU dynticks counter is going to be merged into the context trackingsubsystem. Start with moving the idle extended quiescent statesentrypoints to context tracking. For now those are dumb redirections toexisting RCU calls.[ paulmck: Apply kernel test robot feedback. ]Signed-off-by: Frederic Weisbecker Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Neeraj Upadhyay Cc: Uladzislau Rezki Cc: Joel Fernandes Cc: Boqun Feng Cc: Nicolas Saenz Julienne Cc: Marcelo Tosatti Cc: Xiongfeng Wang Cc: Yu Liao Cc: Phil Auld Cc: Paul GortmakerCc: Alex Belits Signed-off-by: Paul E. McKenney Reviewed-by: Nicolas Saenz Julienne Tested-by: Nicolas Saenz Julienne List of files: /linux-6.15/kernel/rcu/Kconfig

9621fbee - rcu: Move expedited grace period (GP) work to RT kthread_worker

Sat, 09 Apr 2022 00:35:27 +0000

rcu: Move expedited grace period (GP) work to RT kthread_workerEnabling CONFIG_RCU_BOOST did not reduce RCU expedited grace-periodlatency because its workqueues run at SCHED_OTHER, and thus can bedelayed by normal processes. This commit avoids these delays by movingthe expedited GP work items to a real-time-priority kthread_worker.This option is controlled by CONFIG_RCU_EXP_KTHREAD and disabled bydefault on PREEMPT_RT=y kernels which disable expedited grace periodsafter boot by unconditionally setting rcupdate.rcu_normal_after_boot=1.The results were evaluated on arm64 Android devices (6GB ram) running5.10 kernel, and capturing trace data in critical user-level code.The table below shows the resulting order-of-magnitude improvementsin synchronize_rcu_expedited() latency:------------------------------------------------------------------------| | workqueues | kthread_worker | Diff |------------------------------------------------------------------------| Count | 725 | 688 | |------------------------------------------------------------------------| Min Duration (ns) | 326 | 447 | 37.12% |------------------------------------------------------------------------| Q1 (ns) | 39,428 | 38,971 | -1.16% |------------------------------------------------------------------------| Q2 - Median (ns) | 98,225 | 69,743 | -29.00% |------------------------------------------------------------------------| Q3 (ns) | 342,122 | 126,638 | -62.98% |------------------------------------------------------------------------| Max Duration (ns) | 372,766,967 | 2,329,671 | -99.38% |------------------------------------------------------------------------| Avg Duration (ns) | 2,746,353 | 151,242 | -94.49% |------------------------------------------------------------------------| Standard Deviation (ns) | 19,327,765 | 294,408 | |------------------------------------------------------------------------The below table show the range of maximums/minimums forsynchronize_rcu_expedited() latency from all experiments:------------------------------------------------------------------------| | workqueues | kthread_worker | Diff |------------------------------------------------------------------------| Total No. of Experiments | 25 | 23 | |------------------------------------------------------------------------| Largest Maximum (ns) | 372,766,967 | 2,329,671 | -99.38% |------------------------------------------------------------------------| Smallest Maximum (ns) | 38,819 | 86,954 | 124.00% |------------------------------------------------------------------------| Range of Maximums (ns) | 372,728,148 | 2,242,717 | |------------------------------------------------------------------------| Largest Minimum (ns) | 88,623 | 27,588 | -68.87% |------------------------------------------------------------------------| Smallest Minimum (ns) | 326 | 447 | 37.12% |------------------------------------------------------------------------| Range of Minimums (ns) | 88,297 | 27,141 | |------------------------------------------------------------------------Cc: "Paul E. McKenney" Cc: Tejun Heo Reported-by: Tim Murray Reported-by: Wei Wang Tested-by: Kyle Lin Tested-by: Chunwei Lu Tested-by: Lulu Wang Signed-off-by: Kalesh Singh Signed-off-by: Paul E. McKenney List of files: /linux-6.15/kernel/rcu/Kconfig

4c3f7b0e - rcutorture: Allow rcutorture without RCU Tasks Rude

Thu, 17 Mar 2022 23:16:45 +0000

rcutorture: Allow rcutorture without RCU Tasks RudeUnless a kernel builds rcutorture, whether built-in or as a module, thatkernel is also built with CONFIG_TASKS_RUDE_RCU, whether anything elseneeds Tasks Rude RCU or not. This unnecessarily increases kernel size.This commit therefore decouples the presence of rcutorture from thepresence of RCU Tasks Rude.However, there is a need to select CONFIG_TASKS_RUDE_RCU for testingpurposes. Except that casual users must not be bothered withquestions -- for them, this needs to be fully automated. There isthus a CONFIG_FORCE_TASKS_RUDE_RCU that selects CONFIG_TASKS_RUDE_RCU,is user-selectable, but which depends on CONFIG_RCU_EXPERT.[ paulmck: Apply kernel test robot feedback. ]Signed-off-by: Paul E. McKenney List of files: /linux-6.15/kernel/rcu/Kconfig

3b6e1dd4 - rcutorture: Allow rcutorture without RCU Tasks

Thu, 17 Mar 2022 22:18:27 +0000

rcutorture: Allow rcutorture without RCU TasksCurrently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU forRCU Tasks. Unless that kernel builds rcutorture, whether built-in or asa module, in which case RCU Tasks is (unnecessarily) used. This bothincreases kernel size and increases the complexity of certain tracingoperations. This commit therefore decouples the presence of rcutorturefrom the presence of RCU Tasks.However, there is a need to select CONFIG_TASKS_RCU for testing purposes.Except that casual users must not be bothered with questions -- for them,this needs to be fully automated. There is thus a CONFIG_FORCE_TASKS_RCUthat selects CONFIG_TASKS_RCU, is user-selectable, but which dependson CONFIG_RCU_EXPERT.[ paulmck: Apply kernel test robot feedback. ]Signed-off-by: Paul E. McKenney List of files: /linux-6.15/kernel/rcu/Kconfig