| 3166bb7c | 30-Nov-2023 |
Mark Johnston <[email protected]> |
ossl: Add AES-GCM support for NEON-enabled armv7
This provides substantially higher throughput than the fallback
implementation.
Reviewed by: jhb
MFC after: 3 months
Sponsored by: Klara, Inc.
Spons
ossl: Add AES-GCM support for NEON-enabled armv7
This provides substantially higher throughput than the fallback
implementation.
Reviewed by: jhb
MFC after: 3 months
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D41305
(cherry picked from commit 629a72376d51aad812d6f1279403bc81c38c35d2)
show more ...
|
| 118b866d | 29-Nov-2023 |
Mark Johnston <[email protected]> |
ossl: Fix some bugs in the fallback AES-GCM implementation
gcm_*_aesni() are used when the AVX512 implementation is not available.
Fix two bugs which manifest when handling operations spanning multi
ossl: Fix some bugs in the fallback AES-GCM implementation
gcm_*_aesni() are used when the AVX512 implementation is not available.
Fix two bugs which manifest when handling operations spanning multiple
segments:
- Avoid underflow when the length of the input is smaller than the
residual.
- In gcm_decrypt_aesni(), ensure that we begin the operation at the
right offset into the input and output buffers.
Reviewed by: jhb
Fixes: 9b1d87286c78 ("ossl: Add a fallback AES-GCM implementation using AES-NI")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D42838
(cherry picked from commit 47d767dab54895f3ba8abac6ab2295797394659e)
show more ...
|
| 84ef0a84 | 29-Nov-2023 |
Mark Johnston <[email protected]> |
ossl: Keep mutable AES-GCM state on the stack
ossl(4)'s AES-GCM implementation keeps mutable state in the session
structure, together with the key schedule. This was done for
convenience, as both a
ossl: Keep mutable AES-GCM state on the stack
ossl(4)'s AES-GCM implementation keeps mutable state in the session
structure, together with the key schedule. This was done for
convenience, as both are initialized together. However, some OCF
consumers, particularly ZFS, assume that requests may be dispatched to
the same session in parallel. Without serialization, this results in
incorrect output.
Fix the problem by explicitly copying per-session state onto the stack
at the beginning of each operation.
PR: 275306
Reviewed by: jhb
Fixes: 9a3444d91c70 ("ossl: Add a VAES-based AES-GCM implementation for amd64")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D42783
(cherry picked from commit 5c0dac0b7a012f326edab06ad85aee5ad68ff120)
show more ...
|
| 9b1d8728 | 02-Jun-2023 |
Mark Johnston <[email protected]> |
ossl: Add a fallback AES-GCM implementation using AES-NI
This lets one use ossl(4) for AES-GCM operations on contemporary amd64
platforms. A kernel benchmark indicates that this gives roughly
equiv
ossl: Add a fallback AES-GCM implementation using AES-NI
This lets one use ossl(4) for AES-GCM operations on contemporary amd64
platforms. A kernel benchmark indicates that this gives roughly
equivalent throughput to aesni(4) for various buffer sizes.
Bulk processing is done in aesni-gcm-x86_64.S, the rest is handled in a
C wrapper ported from OpenSSL's gcm128.c.
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Reviewed by: jhb
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D39967
show more ...
|
| 9a3444d9 | 02-Jun-2023 |
Mark Johnston <[email protected]> |
ossl: Add a VAES-based AES-GCM implementation for amd64
aes-gcm-avx512.S is generated from OpenSSL 3.1 and implements AES-GCM.
ossl_x86.c detects whether the CPU implements the required AVX512
instr
ossl: Add a VAES-based AES-GCM implementation for amd64
aes-gcm-avx512.S is generated from OpenSSL 3.1 and implements AES-GCM.
ossl_x86.c detects whether the CPU implements the required AVX512
instructions; if not, the ossl(4) module does not provide an AES-GCM
implementation. The VAES implementation increases throughput for all
buffer sizes in both directions, up to 2x for sufficiently large
buffers.
The "process" implementation is in two parts: a generic OCF layer in
ossl_aes.c that calls a set of MD functions to do the heavy lifting.
The intent there is to make it possible to add other implementations for
other platforms, e.g., to reduce the diff required for D37421.
A follow-up commit will add a fallback path to legacy AES-NI, so that
ossl(4) can be used in preference to aesni(4) on all amd64 platforms.
In the long term we would like to replace aesni(4) and armv8crypto(4)
with ossl(4).
Note, currently this implementation will not be selected by default
since aesni(4) and ossl(4) return the same probe priority for crypto
sessions, and the opencrypto framework selects the first registered
implementation to break a tie. Since aesni(4) is compiled into the
kernel, aesni(4) wins. A separate change may modify ossl(4) to have
priority.
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Reviewed by: jhb
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D39783
show more ...
|