[TLS] fix spurious warning trace (fixes #3182)

(thx flynn)

x-ref:
"Test config reports invalid ssl.pemfile in $HTTP["host"] condition"
https://redmine.lighttpd.net/issues/3182

[multiple] employ ck_calloc, ck_malloc shared code

employ ck_calloc(), ck_malloc() shared code to slightly reduce code size
(centralize the ck_assert() to check that memory allocation succeeded)

[multiple] mark mod_*_plugin_init() funcs cold

[TLS] simplify TLS config; remove deprecated opts

simplify TLS config; remove deprecated options

These scheduled lighttpd behavior changes have been announced over
the past year.

lighttpd aims to

[TLS] simplify TLS config; remove deprecated opts

simplify TLS config; remove deprecated options

These scheduled lighttpd behavior changes have been announced over
the past year.

lighttpd aims to provide reasonably secure TLS configuration defaults,
and to periodically review and update TLS configuration defaults.
Doing so reduces the need for distros, packagers, and end-users to
specify their own TLS config customizations, which may then be neglected
or cargo-culted far into the future, instead of being periodically
updated to use stronger defaults.

x-ref:
https://wiki.lighttpd.net/Docs_SSL

show more ...

[TLS] upgrade default cipher list to stronger set

upgrade default cipher list to stronger set, changing default from
"HIGH" to "EECDH+AESGCM:AES256+EECDH:CHACHA20:!SHA1:!SHA256:!SHA384"

openssl cip

[TLS] upgrade default cipher list to stronger set

upgrade default cipher list to stronger set, changing default from
"HIGH" to "EECDH+AESGCM:AES256+EECDH:CHACHA20:!SHA1:!SHA256:!SHA384"

openssl ciphers 'EECDH+AESGCM:AES256+EECDH:CHACHA20:!SHA1:!SHA256:!SHA384'
expands to
"TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-CCM8:ECDHE-ECDSA-AES256-CCM:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:PSK-CHACHA20-POLY1305"

Most of these ciphers are widely supported and have been for many years.
These scheduled lighttpd behavior changes have been announced over
the past year.

Providing a strong default cipher list reduces the need for distros,
packagers, and end-users to specify their own cipher lists, which may
then be neglected or cargo-culted far into the future, instead of being
periodically updated to use stronger defaults.

x-ref:
https://wiki.lighttpd.net/Docs_SSL

show more ...

[TLS] handle '+' on ssl-conf-cmd "Options"

[mod_gnutls] use gnutls_record_send_file() if KTLS

use gnutls_record_send_file() if KTLS available

For HTTP/1.x responses streaming files from disk,
gnutls_record_send_file() (when KTLS is availabl

[mod_gnutls] use gnutls_record_send_file() if KTLS

use gnutls_record_send_file() if KTLS available

For HTTP/1.x responses streaming files from disk,
gnutls_record_send_file() (when KTLS is available) might
reduce CPU usage and copying between userspace and kernel.

Note: KTLS is disabled by default in gnutls global config

https://www.gnutls.org/manual/gnutls.html#Enabling-KTLS
"When GnuTLS is build with -–enable-ktls configuration, KTLS is disabled by default. This can be enabled by setting ktls = true in [global] section."
https://www.gnutls.org/manual/gnutls.html#System_002dwide-configuration-of-the-library
"The location of the default configuration file is /etc/gnutls/config, but its actual location may be overridden during compile time or at run-time using the GNUTLS_SYSTEM_PRIORITY_FILE environment variable. The file used can be queried using gnutls_get_system_config_file."

(e.g. on Fedora 37: /etc/crypto-policies/back-ends/gnutls.config)

show more ...

[TLS] try DER format if reading PEM format fails

try DER format if reading PEM format fails for ssl.pemfile, ssl.privkey

[TLS] ssl.openssl.ssl-conf-cmd "DHParameters"

support "DHParameters" in ssl.openssl.ssl-conf-cmd
(replacement for ssl.dh-file)

isolate code setting DHParameters into its own subroutine (code reuse)

[TLS] ssl.openssl.ssl-conf-cmd "DHParameters"

support "DHParameters" in ssl.openssl.ssl-conf-cmd
(replacement for ssl.dh-file)

isolate code setting DHParameters into its own subroutine (code reuse)

Note: TLS library defaults should be preferred over specifying DH params

show more ...

[TLS] inherit ssl.engine from global scope

Since lighttpd 1.4.56, an oversight in config processing missed
setting explicitly p->conf.ssl_enabled = 0 in network.c when
initializing conditions. When

[TLS] inherit ssl.engine from global scope

Since lighttpd 1.4.56, an oversight in config processing missed
setting explicitly p->conf.ssl_enabled = 0 in network.c when
initializing conditions. When ssl.engine = "enable" in lighttpd.conf
global scope, the missing reset in network.c required non-TLS ports
(e.g. $SERVER["socket"] == ":80") to contain ssl.engine = "disable"
in order for requests to those ports to be served rather than erroring.

(This error was discovered during collaboration with jens-maus in
https://github.com/jens-maus/RaspberryMatic/pull/1847)

There have been zero other instances of this error reported since the
release of lighttpd 1.4.56 in Nov 2020.

Therefore, having ssl.engine = "enable" inherited from the global scope
is unlikely to have any widespread impact in practice, and enabling
ssl.engine = "enable" (along with TLS certificate configuration) is now
recommended as default. When ssl.engine = "enable" in the global scope,
ssl.engine = "disable" should be specified in those $SERVER["socket"]
conditions where clear-text is desired.

show more ...

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_i

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_in on keep-alive requests
(affects case where client pipelines HTTP/1.1 requests)

remove con->bytes_read and con->bytes_written
(no longer needed since request_st was split from connection struct
and request bytes_read_ckpt and bytes_written_ckpt are maintained
for HTTP/1.x bytes_in and bytes_out accounting. Also, further back,
chunkqueue internal accounting was simplified to maintain bytes_in
and bytes_out to always match chunkqueue length)

show more ...

[multiple] use buffer_append_char()

[multiple] reset http vers, avoid rare crash (fixes #3152)

(thx ultimator)

do not set r->http_version to HTTP_VERSION_2 when selecting TLS ALPN
if r->handler_module already set, since handler modul

[multiple] reset http vers, avoid rare crash (fixes #3152)

(thx ultimator)

do not set r->http_version to HTTP_VERSION_2 when selecting TLS ALPN
if r->handler_module already set, since handler module is likely
mod_sockproxy, and con->h2 will not get initialized.

This does continue to select "h2", so the mod_sockproxy backend
should be prepared to receive the HTTP/2 client connection preface.

x-ref:
"Random Segfaults with version 1.4.64 w/ mod_sockproxy and ALPN h2"
https://redmine.lighttpd.net/issues/3152

show more ...

[multiple] limit scope of socket config options

warn if socket config options used only at startup are used outside
global scope or $SERVER["socket"] with '==' condition

[TLS] warn if leaf cert read is inactive/expired

When reading certificates, warn if leaf certificate is inactive/expired
(according to notBefore, notAfter fields of leaf certificate)
(note: not addi

[TLS] warn if leaf cert read is inactive/expired

When reading certificates, warn if leaf certificate is inactive/expired
(according to notBefore, notAfter fields of leaf certificate)
(note: not adding a delta for fudge factor when comparing times)
(note: not currently verifying each certificate in chain)

show more ...

[TLS] consistent debug.log-ssl-noise config type

(thx flynn)

x-ref:
https://redmine.lighttpd.net/issues/3146#note-26

[multiple] permit UTF-8 in SSL_CLIENT_S_DN_*

permit non-ASCII UTF-8 in SSL_CLIENT_S_DN_*

x-ref:
https://github.com/ARMmbed/mbedtls/pull/3326#issuecomment-1013921672

[mod_gnutls] lift size check out of DN loop

lift size check out of client Subject DN loop

[multiple] remove buffer_init_buffer()

remove (minor) convenience func; easy to replace

Like buffer_init_string(), buffer_init_buffer() was used in only a few
places at startup or in cold funcs, so

[multiple] remove buffer_init_buffer()

remove (minor) convenience func; easy to replace

Like buffer_init_string(), buffer_init_buffer() was used in only a few
places at startup or in cold funcs, so better off removed from buffer.c

show more ...

[core] quiet compiler warnings

cast away signedness warning in request_check_hostname()
mod_gnutls https_add_ssl_client_entries crts allocated if crt_size != 0
(which is already checked earlier in

[core] quiet compiler warnings

cast away signedness warning in request_check_hostname()
mod_gnutls https_add_ssl_client_entries crts allocated if crt_size != 0
(which is already checked earlier in routine)

report from FaceBook Infer static analysis tool (https://fbinfer.com/)
- quiet dead store warnings
- check return != NULL from allocation funcs

show more ...

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and Fr

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
$ lighttpd -V
+ Y2038 support # Y2038-SAFE
$ lighttpd -V
- Y2038 support (unsafe 32-bit signed time_t) # Y2038-UNSAFE

* Y2038: general platform info
* Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
- all major 64-bit platforms (known to this author) use 64-bit time_t
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
- Linux x32 ABI (different from i686)
- FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
- NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
- OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
- Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
- MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
- e.g. OpenVMS (unknown if lighttpd builds on this platform)
* Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
- Linux 32-bit (including i686)
- glibc 32-bit library support not yet available for 64-bit time_t
- https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
- Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
- https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
"Note: at this point, 64-bit time support in dual-time
configurations is work-in-progress, so for these
configurations, the public API only makes the 32-bit time
support available. In a later change, the public API will
allow user code to choose the time size for a given
compilation unit."
- compiling with -D_TIME_BITS=64 currently has no effect
- glibc recent (Jul 2021) mailing list discussion
- https://public-inbox.org/bug-gnulib/[email protected]/T/
- FreeBSD i386
- DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
* lighttpd prefers system monotonic clock instead of realtime clock
in places where realtime clock is not required
* lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
* (lighttpd presumes that lighttpd will not encounter dates before 1970
during normal operation.)
* lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
to convert negative timestamps for comparisions with 64-bit timestamps
(treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
* lighttpd provides unix_time64_t (int64_t) and
* lighttpd provides struct unix_timespec64 (unix_timespec64_t)
(struct timespec equivalent using unix_time64_t tv_sec member)
* lighttpd provides gmtime64_r() and localtime64_r() wrappers
for platforms 32-bit platforms using 32-bit time_t and
lighttpd temporarily shifts the year in order to use
gmtime_r() and localtime_r() (or gmtime() and localtime())
from standard libraries, before readjusting year and passing
struct tm to formatting functions such as strftime()
* lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
on 32-bit platforms using 32-bit signed time_t, underlying libraries and
underlying filesystems might not behave properly after 32-bit signed time_t
overflows (19 Jan 2038 03:14:08 GMT). If a given 32-bit OS does not work
properly using negative time_t values, then lighttpd likely will not work
properly on that system.

* Other references and blogs
- https://en.wikipedia.org/wiki/Year_2038_problem
- https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
- http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/

show more ...

[multiple] buffer_copy_string_len_lc()

convenience wrapper combining
buffer_copy_string_len()
buffer_to_lower()
and making a single pass over string

[multiple] reduce redundant NULL buffer checks

This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot co

[multiple] reduce redundant NULL buffer checks

This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.

Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.

In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func. In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.

- check for empty strings at config time and set value to NULL if blank
string will be ignored at runtime; at runtime, simple pointer check
for NULL can be used to check for a value that has been set and is not
blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
and use buffer_is_unset() instead of buffer_is_empty(),
where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
truncate string, and use buffer_extend() to extend

Examples where buffer known not to be NULL:
- cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
(though we might set it to NULL if buffer_is_blank(cpv->v.b))
- address of buffer is arg (&foo)
(compiler optimizer detects this in most, but not all, cases)
- buffer is checked for NULL earlier in func
- buffer is accessed in same scope without a NULL check (e.g. b->ptr)

internal behavior change:
callers must not pass a NULL buffer to some funcs.
- buffer_init_buffer() requires non-null args
- buffer_copy_buffer() requires non-null args
- buffer_append_string_buffer() requires non-null args
- buffer_string_space() requires non-null arg

show more ...

[TLS] reset stek_rotate_ts if clock moves backward

reset stek_rotate_ts if clock moves backwards > 28800 seconds

x-ref:
"Lighttpd 1.4.58 SSL connections stop working if system time of lighttpd se

[TLS] reset stek_rotate_ts if clock moves backward

reset stek_rotate_ts if clock moves backwards > 28800 seconds

x-ref:
"Lighttpd 1.4.58 SSL connections stop working if system time of lighttpd server is changed to future one (+12h or even days)"
https://redmine.lighttpd.net/issues/3080

show more ...

[TLS] write_cq_ssl defer remove_finished_chunks

not expecting 0-length chunks, but handle within loops as cold path

mark some cold paths in read_cq_ssl and write_cq_ssl callback funcs