Changes in Makefile

511484fa - riscv/crc64: add Zbc optimized CRC64 functions

Sun, 16 Feb 2025 22:55:30 +0000

riscv/crc64: add Zbc optimized CRC64 functionsWire up crc64_be_arch() and crc64_nvme_arch() for 64-bit RISC-V usingcrc-clmul-template.h. This greatly improves the performance of theseCRCs on Zbc-capable CPUs in 64-bit kernels.These optimized CRC64 functions are not yet supported in 32-bit kernels,since crc-clmul-template.h assumes that the CRC fits in an unsignedlong. That implementation limitation could be addressed, but it wouldadd a fair bit of complexity, so it has been omitted for now.Tested-by: Björn Töpel Acked-by: Alexandre Ghiti Link: https://lore.kernel.org/r/20250216225530.306980-5-ebiggers@kernel.orgSigned-off-by: Eric Biggers List of files: /linux-6.15/arch/riscv/lib/Makefile

8bf3e178 - riscv/crc-t10dif: add Zbc optimized CRC-T10DIF function

Sun, 16 Feb 2025 22:55:29 +0000

riscv/crc-t10dif: add Zbc optimized CRC-T10DIF functionWire up crc_t10dif_arch() for RISC-V using crc-clmul-template.h. Thisgreatly improves CRC-T10DIF performance on Zbc-capable CPUs.Tested-by: Björn Töpel Acked-by: Alexandre Ghiti Link: https://lore.kernel.org/r/20250216225530.306980-4-ebiggers@kernel.orgSigned-off-by: Eric Biggers List of files: /linux-6.15/arch/riscv/lib/Makefile

72acff5f - riscv/crc32: reimplement the CRC32 functions using new template

Sun, 16 Feb 2025 22:55:28 +0000

riscv/crc32: reimplement the CRC32 functions using new templateDelete the previous Zbc optimized CRC32 code, and re-implement it usingthe new template. The new implementation is more optimized and sharesmore code among CRC variants.Tested-by: Björn Töpel Acked-by: Alexandre Ghiti Link: https://lore.kernel.org/r/20250216225530.306980-3-ebiggers@kernel.orgSigned-off-by: Eric Biggers List of files: /linux-6.15/arch/riscv/lib/Makefile

d36cebe0 - lib/crc32: improve support for arch-specific overrides

Mon, 02 Dec 2024 01:08:27 +0000

lib/crc32: improve support for arch-specific overridesCurrently the CRC32 library functions are defined as weak symbols, andthe arm64 and riscv architectures override them.This method of arch-specific overrides has the limitation that it onlyworks when both the base and arch code is built-in. Also, it makes thearch-specific code be silently not used if it is accidentally built withlib-y instead of obj-y; unfortunately the RISC-V code does this.This commit reorganizes the code to have explicit *_arch() functionsthat are called when they are enabled, similar to how some of the cryptolibrary code works (e.g. chacha_crypt() calls chacha_crypt_arch()).Make the existing kconfig choice for the CRC32 implementation alsocontrol whether the arch-optimized implementation (if one is available)is enabled or not. Make it enabled by default if CRC32 is also enabled.The result is that arch-optimized CRC32 library functions will beincluded automatically when appropriate, but it is now possible todisable them. They can also now be built as a loadable module if theCRC32 library functions happen to be used only by loadable modules, inwhich case the arch and base CRC32 modules will be automatically loadedvia direct symbol dependency when appropriate.Reviewed-by: Ard Biesheuvel Link: https://lore.kernel.org/r/20241202010844.144356-3-ebiggers@kernel.orgSigned-off-by: Eric Biggers List of files: /linux-6.15/arch/riscv/lib/Makefile

58ff5371 - riscv: Omit optimized string routines when using KASAN

Thu, 01 Aug 2024 03:36:59 +0000

riscv: Omit optimized string routines when using KASANThe optimized string routines are implemented in assembly, so they arenot instrumented for use with KASAN. Fall back to the C version of theroutines in order to improve KASAN coverage. This fixes thekasan_strings() unit test.Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti Tested-by: Alexandre Ghiti Link: https://lore.kernel.org/r/20240801033725.28816-2-samuel.holland@sifive.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

a43fe27d - riscv: Optimize crc32 with Zbc extension

Fri, 21 Jun 2024 05:47:07 +0000

riscv: Optimize crc32 with Zbc extensionAs suggested by the B-ext spec, the Zbc (carry-less multiplication)instructions can be used to accelerate CRC calculations. Currently, thecrc32 is the most widely used crc function inside kernel, so this patchfocuses on the optimization of just the crc32 APIs.Compared with the current table-lookup based optimization, Zbc basedoptimization can also achieve large stride during CRC calculation loop,meantime, it avoids the memory access latency of the table-lookup basedimplementation and it reduces memory footprint.If Zbc feature is not supported in a runtime environment, then thetable-lookup based implementation would serve as fallback via alternativemechanism.By inspecting the vmlinux built by gcc v12.2.0 with default optimizationlevel (-O2), we can see below instruction count change for each 8-bytestride in the CRC32 loop:rv64: crc32_be (54->31), crc32_le (54->13), __crc32c_le (54->13)rv32: crc32_be (50->32), crc32_le (50->16), __crc32c_le (50->16)The compile target CPU is little endian, extra effort is needed for byteswapping for the crc32_be API, thus, the instruction count change is notas significant as that in the *_le cases.This patch is tested on QEMU VM with the kernel CRC32 selftest for bothrv64 and rv32. Running the CRC32 selftest on a real hardware (SpacemiT K1)with Zbc extension shows 65% and 125% performance improvement respectivelyon crc32_test() and crc32c_test().Signed-off-by: Xiao Wang Reviewed-by: Charlie Jenkins Link: https://lore.kernel.org/r/20240621054707.1847548-1-xiao.w.wang@intel.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

a04c192e - riscv: Add checksum library

Mon, 08 Jan 2024 23:57:05 +0000

riscv: Add checksum libraryProvide a 32 and 64 bit version of do_csum. When compiled for 32-bitwill load from the buffer in groups of 32 bits, and when compiled for64-bit will load in groups of 64 bits.Additionally provide riscv optimized implementation of csum_ipv6_magic.Signed-off-by: Charlie Jenkins Acked-by: Conor Dooley Reviewed-by: Xiao Wang Link: https://lore.kernel.org/r/20240108-optimize_checksum-v15-4-1c50de5f2167@rivosinc.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

c2a658d4 - riscv: lib: vectorize copy_to_user/copy_from_user

Mon, 15 Jan 2024 05:59:24 +0000

riscv: lib: vectorize copy_to_user/copy_from_userThis patch utilizes Vector to perform copy_to_user/copy_from_user. IfVector is available and the size of copy is large enough for Vector toperform better than scalar, then direct the kernel to do Vector copiesfor userspace. Though the best programming practice for users is toreduce the copy, this provides a faster variant when copies areinevitable.The optimal size for using Vector, copy_to_user_thres, is only aheuristic for now. We can add DT parsing if people feel the need ofcustomizing it.The exception fixup code of the __asm_vector_usercopy must fallback tothe scalar one because accessing user pages might fault, and must besleepable. Current kernel-mode Vector does not allow tasks to bepreemptible, so we must disactivate Vector and perform a scalar fallbackin such case.The original implementation of Vector operations comes fromhttps://github.com/sifive/sifive-libc, which we agree to contribute toLinux kernel.Co-developed-by: Jerry Shih Signed-off-by: Jerry Shih Co-developed-by: Nick Knight Signed-off-by: Nick Knight Suggested-by: Guo Ren Signed-off-by: Andy Chiu Tested-by: Björn Töpel Tested-by: Lad Prabhakar Link: https://lore.kernel.org/r/20240115055929.4736-6-andy.chiu@sifive.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

c5674d00 - riscv: Add vector extension XOR implementation

Mon, 15 Jan 2024 05:59:22 +0000

riscv: Add vector extension XOR implementationThis patch adds support for vector optimized XOR and it is tested inqemu.Co-developed-by: Han-Kuan Chen Signed-off-by: Han-Kuan Chen Signed-off-by: Greentime Hu Signed-off-by: Andy Chiu Tested-by: Björn Töpel Tested-by: Lad Prabhakar Link: https://lore.kernel.org/r/20240115055929.4736-4-andy.chiu@sifive.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

ab0f7746 - RISC-V: Use Zicboz in clear_page when available

Fri, 24 Feb 2023 16:26:29 +0000

RISC-V: Use Zicboz in clear_page when availableUsing memset() to zero a 4K page takes 563 total instructions, where20 are branches. clear_page(), with Zicboz and a 64 byte block size,takes 169 total instructions, where 4 are branches and 33 are nops.Even though the block size is a variable, thanks to alternatives, wecan still implement a Duff device without having to do any preliminarycalculations. This is achieved by using the alternatives' cpufeaturevalue (the upper 16 bits of patch_id). The value used is the maximumzicboz block size order accepted at the patch site. This enables usto stop patching / unrolling when 4K bytes have been zeroed (we wouldloop and continue after 4K if the page size would be larger)For 4K pages, unrolling 16 times allows block sizes of 64 and 128 toonly loop a few times and larger block sizes to not loop at all. Sincecbo.zero doesn't take an offset, we also need an 'add' after eachinstruction, making the loop body 112 to 160 bytes. Hopefully thisis small enough to not cause icache misses.Signed-off-by: Andrew Jones Acked-by: Conor Dooley Link: https://lore.kernel.org/r/20230224162631.405473-7-ajones@ventanamicro.comSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

56e0790c - RISC-V: add infrastructure to allow different str* implementations

Fri, 13 Jan 2023 21:23:00 +0000

RISC-V: add infrastructure to allow different str* implementationsDepending on supported extensions on specific RISC-V cores,optimized str* functions might make sense.This adds basic infrastructure to allow patching the function callsvia alternatives later on.The Linux kernel provides standard implementations for string functionsbut when architectures want to extend them, they need to provide theirown.The added generic string functions are done in assembler (taken fromdisassembling the main-kernel functions for now) to allow us to controlthe used registers and extend them with optimized variants.This doesn't override the compiler's use of builtin replacements. So stillfirst of all the compiler will select if a builtin will be better suitablei.e. for known strings. For all regular cases we will want to laterselect possible optimized variants and in the worst case fall back to thegeneric implemention added with this change.Reviewed-by: Andrew Jones Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230113212301.3534711-2-heiko@sntech.deSigned-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

ee55ff80 - riscv: Add support for function error injection

Thu, 17 Dec 2020 16:01:45 +0000

riscv: Add support for function error injectionInspired by the commit 42d038c4fb00 ("arm64: Add support for functionerror injection"), this patch supports function error injection forriscv.This patch mainly support two functions: one is regs_set_return_value()which is used to overwrite the return value; the another function isoverride_function_with_return() which is to override the probedfunction returning and jump to its caller.Test log: cd /sys/kernel/debug/fail_function echo sys_clone > inject echo 100 > probability echo 1 > interval ls /[ 313.176875] FAULT_INJECTION: forcing a failure.[ 313.176875] name fail_function, interval 1, probability 100, space 0, times 1[ 313.184357] CPU: 0 PID: 87 Comm: sh Not tainted 5.8.0-rc5-00007-g6a758cc #117[ 313.187616] Call Trace:[ 313.189100] [] walk_stackframe+0x0/0xc2[ 313.191626] [] show_stack+0x40/0x4c[ 313.193927] [] dump_stack+0x7c/0x96[ 313.194795] [] should_fail+0x140/0x142[ 313.195923] [] fei_kprobe_handler+0x2c/0x5a[ 313.197687] [] kprobe_breakpoint_handler+0xb4/0x18a[ 313.200054] [] do_trap_break+0x36/0xca[ 313.202147] [] ret_from_exception+0x0/0xc[ 313.204556] [] ret_from_syscall+0x0/0x2-sh: can't fork: Invalid argumentSigned-off-by: Guo Ren Reviewed-by: Masami Hiramatsu Cc: Palmer Dabbelt Cc: Paul Walmsley Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

04091d6c - riscv: provide memmove implementation

Mon, 30 Nov 2020 09:13:19 +0000

riscv: provide memmove implementationThe memmove used by the kernel feature like KASAN.Signed-off-by: Nick Hu Signed-off-by: Nick Hu Signed-off-by: Nylon Chen Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

11129e8e - riscv: use memcpy based uaccess for nommu again

Mon, 07 Sep 2020 05:58:22 +0000

riscv: use memcpy based uaccess for nommu againThis reverts commit adccfb1a805ea84d2db38eb53032533279bdaa97.Now that the generic uaccess by mempcy code handles unaligned addressesthe generic code can be used for all RISC-V CPUs.Signed-off-by: Christoph Hellwig Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

adccfb1a - riscv: uaccess should be used in nommu mode

Tue, 03 Mar 2020 09:34:17 +0000

riscv: uaccess should be used in nommu modeIt might have the unaligned access exception when trying to exchange datawith user space program. In this case, it failed in tty_ioctl(). Thereforewe should enable uaccess.S for NOMMU mode since the generic code doesn'thandle the unaligned access cases. 0x8013a212 : ld a5,460(s1)[ 0.115279] Oops - load address misaligned [#1][ 0.115284] CPU: 0 PID: 29 Comm: sh Not tainted 5.4.0-rc5-00020-gb4c27160d562-dirty #36[ 0.115294] epc: 000000008013a212 ra : 000000008013a212 sp : 000000008f48dd50[ 0.115303] gp : 00000000801cac28 tp : 000000008fb80000 t0 : 00000000000000e8[ 0.115312] t1 : 000000008f58f108 t2 : 0000000000000009 s0 : 000000008f48ddf0[ 0.115321] s1 : 000000008f8c6220 a0 : 0000000000000001 a1 : 000000008f48dd28[ 0.115330] a2 : 000000008fb80000 a3 : 00000000801a7398 a4 : 0000000000000000[ 0.115339] a5 : 0000000000000000 a6 : 000000008f58f0c6 a7 : 000000000000001d[ 0.115348] s2 : 000000008f8c6308 s3 : 000000008f78b7c8 s4 : 000000008fb834c0[ 0.115357] s5 : 0000000000005413 s6 : 0000000000000000 s7 : 000000008f58f2b0[ 0.115366] s8 : 000000008f858008 s9 : 000000008f776818 s10: 000000008f776830[ 0.115375] s11: 000000008fb840a8 t3 : 1999999999999999 t4 : 000000008f78704c[ 0.115384] t5 : 0000000000000005 t6 : 0000000000000002[ 0.115391] status: 0000000200001880 badaddr: 000000008f8c63ec cause: 0000000000000004[ 0.115401] ---[ end trace 00d490c6a8b6c9ac ]---This failure could be fixed after this patch applied.[ 0.002282] Run /init as init processInitializing random number generator... [ 0.005573] random: dd: uninitialized urandom read (512 bytes read)done.Welcome to Buildrootbuildroot login: rootPassword:Jan 1 00:00:00 login[62]: root login on 'ttySIF0'~ #Signed-off-by: Greentime Hu Reviewed-by: Palmer Dabbelt Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

6bd33e1e - riscv: add nommu support

Mon, 28 Oct 2019 12:10:41 +0000

riscv: add nommu supportThe kernel runs in M-mode without using page tables, and thus can't runbare metal without help from additional firmware.Most of the patch is just stubbing out code not needed without pagetables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack.In addition to enabling the nommu code a new defconfig for a smallkernel image that can run in nommu mode on qemu is also provided, to runa kernel in qemu you can use the following command line:qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0Contains contributions from Damien Le Moal .Signed-off-by: Christoph Hellwig Reviewed-by: Anup Patel [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning]Signed-off-by: Paul Walmsley List of files: /linux-6.15/arch/riscv/lib/Makefile

81a48ee4 - RISC-V: Remove udivdi3

Wed, 07 Aug 2019 00:05:08 +0000

RISC-V: Remove udivdi3This should never have landed in the first place: it was added as partof 64-bit divide support for 32-bit systems, but the kernel doesn'tallow this sort of division. I must have forgotten to remove it.This patch removes the support. Since this routine only worked on64-bit platforms but was only built on 32-bit platforms, it'sessentially just nonsense anyway.Signed-off-by: Palmer Dabbelt Acked-by: Nicolas Pitre Link: https://lore.kernel.org/linux-riscv/nycvar.YSQ.7.76.1908061413360.19480@knanqh.ubzr/T/#tReported-by: Eric Lin Signed-off-by: Paul Walmsley List of files: /linux-6.15/arch/riscv/lib/Makefile

ec8f24b7 - treewide: Add SPDX license identifier - Makefile/Kconfig

Sun, 19 May 2019 12:07:45 +0000

treewide: Add SPDX license identifier - Makefile/KconfigAdd SPDX license identifiers to all Make/Kconfig files which: - Have no license information of any formThese files fall under the project license, GPL v2 only. The resulting SPDXlicense identifier is: GPL-2.0-onlySigned-off-by: Thomas Gleixner Signed-off-by: Greg Kroah-Hartman List of files: /linux-6.15/arch/riscv/lib/Makefile

85d90b91 - RISC-V: lib: Fix build error for 64-bit

Wed, 31 Oct 2018 06:47:07 +0000

RISC-V: lib: Fix build error for 64-bitFixes the following build error from tinyconfig:riscv64-unknown-linux-gnu-ld: kernel/sched/fair.o: in function `.L8':fair.c:(.text+0x70): undefined reference to `__lshrti3'riscv64-unknown-linux-gnu-ld: kernel/time/clocksource.o: in function `.L0 ':clocksource.c:(.text+0x334): undefined reference to `__lshrti3'Fixes: 7f47c73b355f ("RISC-V: Build tishift only on 64-bit")Signed-off-by: Olof Johansson Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile

7f47c73b - RISC-V: Build tishift only on 64-bit

Tue, 02 Oct 2018 08:52:27 +0000

RISC-V: Build tishift only on 64-bitOnly RV64 supports 128 integer size.Signed-off-by: Zong Li Reviewed-by: Christoph Hellwig Signed-off-by: Palmer Dabbelt List of files: /linux-6.15/arch/riscv/lib/Makefile