Changes in inc_unless_negative

1d78814d - locking/atomic: scripts: simplify raw_atomic*() definitions

Mon, 05 Jun 2023 07:01:20 +0000

locking/atomic: scripts: simplify raw_atomic*() definitionsCurrently each ordering variant has several potential definitions,with a mixture of preprocessor and C definitions, including severalcopies of its C prototype, e.g.| #if defined(arch_atomic_fetch_andnot_acquire)| #define raw_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot_acquire| #elif defined(arch_atomic_fetch_andnot_relaxed)| static __always_inline int| raw_atomic_fetch_andnot_acquire(int i, atomic_t *v)| {| int ret = arch_atomic_fetch_andnot_relaxed(i, v);| __atomic_acquire_fence();| return ret;| }| #elif defined(arch_atomic_fetch_andnot)| #define raw_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot| #else| static __always_inline int| raw_atomic_fetch_andnot_acquire(int i, atomic_t *v)| {| return raw_atomic_fetch_and_acquire(~i, v);| }| #endifMake this a bit simpler by defining the C prototype once, and writingthe various potential definitions as plain C code guarded by ifdeffery.For example, the above becomes:| static __always_inline int| raw_atomic_fetch_andnot_acquire(int i, atomic_t *v)| {| #if defined(arch_atomic_fetch_andnot_acquire)| return arch_atomic_fetch_andnot_acquire(i, v);| #elif defined(arch_atomic_fetch_andnot_relaxed)| int ret = arch_atomic_fetch_andnot_relaxed(i, v);| __atomic_acquire_fence();| return ret;| #elif defined(arch_atomic_fetch_andnot)| return arch_atomic_fetch_andnot(i, v);| #else| return raw_atomic_fetch_and_acquire(~i, v);| #endif| }Which is far easier to read. As we now always have a single copy of theC prototype wrapping all the potential definitions, we now have anobvious single location for kerneldoc comments.At the same time, the fallbacks for raw_atomic*_xhcg() are made to use'new' rather than 'i' as the name of the new value. This is what theexisting fallback template used, and is more consistent with theraw_atomic{_try,}cmpxchg() fallbacks.There should be no functional change as a result of this patch.Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Kees Cook Link: https://lore.kernel.org/r/20230605070124.3741859-24-mark.rutland@arm.com List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

9257959a - locking/atomic: scripts: restructure fallback ifdeffery

Mon, 05 Jun 2023 07:01:17 +0000

locking/atomic: scripts: restructure fallback ifdefferyCurrently the various ordering variants of an atomic operation aredefined in groups of full/acquire/release/relaxed ordering variants withsome shared ifdeffery and several potential definitions of each orderingvariant in different branches of the shared ifdeffery.As an ordering variant can have several potential definitions downdifferent branches of the shared ifdeffery, it can be painful for ahuman to find a relevant definition, and we don't have a good locationto place anything common to all definitions of an ordering variant (e.g.kerneldoc).Historically the grouping of full/acquire/release/relaxed orderingvariants was necessary as we filled in the missing atomics in the samenamespace as the architecture used. It would be easy to accidentallydefine one ordering fallback in terms of another ordering fallback withredundant barriers, and avoiding that would otherwise require a lot ofbaroque ifdeffery.With recent changes we no longer need to fill in the missing atomics inthe arch_atomic*_() namespace, and only need to fill in theraw_atomic*_() namespace. Due to this, there's no risk of anamespace collision, and we can define each raw_atomic*_ orderingvariant with its own ifdeffery checking for the arch_atomic*_ordering variants.Restructure the fallbacks in this way, with each ordering variant havingits own ifdeffery of the form:| #if defined(arch_atomic_fetch_andnot_acquire)| #define raw_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot_acquire| #elif defined(arch_atomic_fetch_andnot_relaxed)| static __always_inline int| raw_atomic_fetch_andnot_acquire(int i, atomic_t *v)| {| int ret = arch_atomic_fetch_andnot_relaxed(i, v);| __atomic_acquire_fence();| return ret;| }| #elif defined(arch_atomic_fetch_andnot)| #define raw_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot| #else| static __always_inline int| raw_atomic_fetch_andnot_acquire(int i, atomic_t *v)| {| return raw_atomic_fetch_and_acquire(~i, v);| }| #endifNote that where there's no relevant arch_atomic*_() orderingvariant, we'll define the operation in terms of a distinctraw_atomic*_(), as this itself might have been filled in with afallback.As we now generate the raw_atomic*_() implementations directly, weno longer need the trivial wrappers, so they are removed.This makes the ifdeffery easier to follow, and will allow for furtherimprovements in subsequent patches.There should be no functional change as a result of this patch.Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Kees Cook Link: https://lore.kernel.org/r/20230605070124.3741859-21-mark.rutland@arm.com List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

f3e615b4 - locking/atomic: remove ARCH_ATOMIC remanants

Tue, 13 Jul 2021 10:52:50 +0000

locking/atomic: remove ARCH_ATOMIC remanantsNow that gen-atomic-fallback.sh is only used to generate the arch_*fallbacks, we don't need to also generate the non-arch_* forms, and canremovethe infrastructure this needed.There is no change to any of the generated headers as a result of thispatch.Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20210713105253.7615-3-mark.rutland@arm.com List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

37f8173d - locking/atomics: Flip fallbacks and instrumentation

Fri, 24 Jan 2020 21:13:03 +0000

locking/atomics: Flip fallbacks and instrumentationCurrently instrumentation of atomic primitives is done at the architecturelevel, while composites or fallbacks are provided at the generic level.The result is that there are no uninstrumented variants of thefallbacks. Since there is now need of such variants to isolate text pokefrom any form of instrumentation invert this ordering.Doing this means moving the instrumentation into the generic code aswell as having (for now) two variants of the fallbacks.Notes: - the various *cond_read* primitives are not proper fallbacks and got moved into linux/atomic.c. No arch_ variants are generated because the base primitives smp_cond_load*() are instrumented. - once all architectures are moved over to arch_atomic_ one of the fallback variants can be removed and some 2300 lines reclaimed. - atomic_{read,set}*() are no longer double-instrumentedReported-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Thomas Gleixner Acked-by: Mark Rutland Link: https://lkml.kernel.org/r/20200505134058.769149955@linutronix.de List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

765dcd20 - asm-generic/atomic: Use __always_inline for fallback wrappers

Tue, 26 Nov 2019 14:04:05 +0000

asm-generic/atomic: Use __always_inline for fallback wrappersUse __always_inline for atomic fallback wrappers. When building for size(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined toinline even relatively small static inline functions that are assumed tobe inlinable such as atomic ops. This can cause problems, for example inUACCESS regions.While the fallback wrappers aren't pure wrappers, they are trivialnonetheless, and the function they wrap should determine the finalinlining policy.For x86 tinyconfig we observe:- vmlinux baseline: 1315988- vmlinux with patch: 1315928 (-60 bytes)[ tglx: Cherry-picked from KCSAN ]Suggested-by: Mark Rutland Signed-off-by: Marco Elver Acked-by: Mark Rutland Signed-off-by: Paul E. McKenney Signed-off-by: Thomas Gleixner List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

944bc9cc - asm-generic/atomic: Use __always_inline for fallback wrappers

Tue, 26 Nov 2019 14:04:05 +0000

asm-generic/atomic: Use __always_inline for fallback wrappersUse __always_inline for atomic fallback wrappers. When building for size(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined toinline even relatively small static inline functions that are assumed tobe inlinable such as atomic ops. This can cause problems, for example inUACCESS regions.While the fallback wrappers aren't pure wrappers, they are trivialnonetheless, and the function they wrap should determine the finalinlining policy.For x86 tinyconfig we observe:- vmlinux baseline: 1315988- vmlinux with patch: 1315928 (-60 bytes)Suggested-by: Mark Rutland Signed-off-by: Marco Elver Acked-by: Mark Rutland Signed-off-by: Paul E. McKenney List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

4d8e5cd2 - locking/atomics: Fix scripts/atomic/ script permissions

Thu, 01 Nov 2018 11:44:48 +0000

locking/atomics: Fix scripts/atomic/ script permissionsMark all these scripts executable.Cc: Mark Rutland Cc: Peter Zijlstra (Intel) Cc: Will Deacon Cc: linux-arm-kernel@lists.infradead.orgCc: Catalin Marinas Cc: linuxdrivers@attotech.comCc: dvyukov@google.comCc: boqun.feng@gmail.comCc: arnd@arndb.deCc: aryabinin@virtuozzo.comCc: glider@google.comCc: linux-kernel@vger.kernel.orgSigned-off-by: Ingo Molnar List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative

ace9bad4 - locking/atomics: Add common header generation files

Tue, 04 Sep 2018 10:48:25 +0000

locking/atomics: Add common header generation filesTo minimize repetition, to allow for future rework, and to ensureregularity of the various atomic APIs, we'd like to automaticallygenerate (the bulk of) a number of headers related to atomics.This patch adds the infrastructure to do so, leaving actual conversionof headers to subsequent patches. This infrastructure consists of:* atomics.tbl - a table describing the functions in the atomics API, with names, prototypes, and metadata describing the variants that exist (e.g fetch/return, acquire/release/relaxed). Note that the return type is dependent on the particular variant.* atomic-tbl.sh - a library of routines useful for dealing with atomics.tbl (e.g. querying which variants exist, or generating argument/parameter lists for a given function variant).* gen-atomic-fallback.sh - a script which generates a header of fallbacks, covering cases where architecture omit certain functions (e.g. omitting relaxed variants).* gen-atomic-long.sh - a script which generates wrappers providing the atomic_long API atomic of the relevant atomic or atomic64 API, ensuring the APIs are consistent.* gen-atomic-instrumented.sh - a script which generates atomic* wrappers atop of arch_atomic* functions, with automatically generated KASAN instrumentation.* fallbacks/* - a set of fallback implementations for atomics, which should be used when no implementation of a given atomic is provided. These are used by gen-atomic-fallback.sh to generate fallbacks, and these are also used by other scripts to determine the set of optional atomics (as required to generate preprocessor guards correctly). Fallbacks may use the following variables: ${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be used to derive the atomic type, and to prefix functions ${int} integer type: int/s64/long ${pfx} variant prefix, e.g. fetch_ ${name} base function name, e.g. add ${sfx} variant suffix, e.g. _return ${order} order suffix, e.g. _relaxed ${atomicname} full name, e.g. atomic64_fetch_add_relaxed ${ret} return type of the function, e.g. void ${retstmt} a return statement (with a trailing space), unless the variant returns void ${params} parameter list for the function declaration, e.g. "int i, atomic_t *v" ${args} argument list for invoking the function, e.g. "i, v" ... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are open-coded for fallbacks where these do not vary, or are critical to understanding the logic of the fallback.The MAINTAINERS entry for the atomic infrastructure is updated to coverthe new scripts.There should be no functional change as a result of this patch.Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Cc: linux-arm-kernel@lists.infradead.orgCc: catalin.marinas@arm.comCc: Will Deacon Cc: linuxdrivers@attotech.comCc: dvyukov@google.comCc: Boqun Feng Cc: arnd@arndb.deCc: aryabinin@virtuozzo.comCc: glider@google.comLink: http://lkml.kernel.org/r/20180904104830.2975-2-mark.rutland@arm.comSigned-off-by: Ingo Molnar List of files: /linux-6.15/scripts/atomic/fallbacks/inc_unless_negative