[multiple] store ptrs to remote addr in request_st (#3192)

adds two pointers to (request_st *) (cost: 16 bytes in 64-bit builds)

prepares for upcoming changes to mod_extforward to manage remote add

[multiple] store ptrs to remote addr in request_st (#3192)

adds two pointers to (request_st *) (cost: 16 bytes in 64-bit builds)

prepares for upcoming changes to mod_extforward to manage remote addr
per request for HTTP/2 requests, rather than remote addr per connection.

Modern load balancers often provide options to reuse connections for
*different* clients, and therefore mod_extforward might change the
remote addr per request.

x-ref:
"RFE: mod_extforward and multiplexed requests via HTTP/2"
https://redmine.lighttpd.net/issues/3192
"Evaluation of remote_addr for mod_maxminddb for multiplexed connections"
https://redmine.lighttpd.net/issues/3191

show more ...

[core] h2: avoid sending small WINDOW_UPDATE frames

Note this may slow down uploads due to delay in sending WINDOW_UPDATE
smaller than the default max frame size (16384), but as a trade-off
this aim

[core] h2: avoid sending small WINDOW_UPDATE frames

Note this may slow down uploads due to delay in sending WINDOW_UPDATE
smaller than the default max frame size (16384), but as a trade-off
this aims to reduce degenerative behavior from clients sending an
increasing number of tiny DATA frames. The default 65535 value for
SETTINGS_INITIAL_WINDOW_SIZE (which lighttpd immediately increases to
65536) is larger than 16384, so deferring small updates should not
exhaust the window (from lighttpd's perspective).

x-ref:
"Slow upload / Increase CPU/Memory usage with HTTP/2 enabled"
https://redmine.lighttpd.net/issues/3089
"libnghttp2 degenerative behavior possible when HTTP/2 window size exhausted"
https://github.com/nghttp2/nghttp2/issues/1722

show more ...

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_i

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_in on keep-alive requests
(affects case where client pipelines HTTP/1.1 requests)

remove con->bytes_read and con->bytes_written
(no longer needed since request_st was split from connection struct
and request bytes_read_ckpt and bytes_written_ckpt are maintained
for HTTP/1.x bytes_in and bytes_out accounting. Also, further back,
chunkqueue internal accounting was simplified to maintain bytes_in
and bytes_out to always match chunkqueue length)

show more ...

[core] reusable code for r->state strings

convenience macros/functions for display purposes

(historical labels from mod_status are preserved for compatibility)

[core] HTTP/2 PRIORITY_UPDATE frame (experimental)

*experimental* support for HTTP/2 PRIORITY_UPDATE frame

x-ref:
"Extensible Prioritization Scheme for HTTP"
https://datatracker.ietf.org/doc/ht

[core] HTTP/2 PRIORITY_UPDATE frame (experimental)

*experimental* support for HTTP/2 PRIORITY_UPDATE frame

x-ref:
"Extensible Prioritization Scheme for HTTP"
https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-priority-12
https://datatracker.ietf.org/doc/html/rfc9218

show more ...

[core] request_config bitmasks for smaller struct

[multiple] WebSockets over HTTP/2 (fixes #3151)

Add support for WebSockets over HTTP/2 to lighttpd core and to
mod_cgi w/ config: cgi.upgrade = "enable"
mod_proxy w/ config: proxy.head

[multiple] WebSockets over HTTP/2 (fixes #3151)

Add support for WebSockets over HTTP/2 to lighttpd core and to
mod_cgi w/ config: cgi.upgrade = "enable"
mod_proxy w/ config: proxy.header += ("upgrade" => "enable")
mod_wstunnel

HTTP/2 CONNECT extension defined in RFC8441 is translated to HTTP/1.1
'Upgrade: websocket' requests to mod_cgi or mod_proxy, and is handled
directly in mod_wstunnel.

x-ref:
WebSockets over HTTP/2
https://redmine.lighttpd.net/issues/3151
Bootstrapping WebSockets with HTTP/2
https://datatracker.ietf.org/doc/html/rfc8441

show more ...

[multiple] remove r->physical.etag

(no longer used; was used as temporary buffer)

[core] fix reqpool mem corruption in 1.4.62 (fixes #3118)

x-ref:
"Segfault after updating to version 1.4.62"
https://redmine.lighttpd.net/issues/3118
"Segfault on closing connections"
https:

[core] fix reqpool mem corruption in 1.4.62 (fixes #3118)

x-ref:
"Segfault after updating to version 1.4.62"
https://redmine.lighttpd.net/issues/3118
"Segfault on closing connections"
https://redmine.lighttpd.net/issues/3119

show more ...

[core] save config regex captures only if used

save config regex captures separately only if used by url.redirect
or url.rewrite replacement directives within the condition
(or for conditions contai

[core] save config regex captures only if used

save config regex captures separately only if used by url.redirect
or url.rewrite replacement directives within the condition
(or for conditions containing directives from any other module
which calls config_capture() for its directives during init)

keep pointer to match data (cond_match_t *) in r->cond_match[]
rather than cond_match_t to reduce data copying in h2_init_stream().
h2_init_stream() copies the results for already-evaluated conditions
to avoid re-evaluating connection-level conditions for each and every
stream. When conditions are reset, then the pointer in r->cond_match[]
is updated when the condition is re-evaluated. (This all assumes that
HTTP/2 connection-level conditions are not unset or re-evaluated once
HTTP/2 streams begin.)

show more ...

[multiple] de-dup file and piped loggers (fixes #3101)

de-dup file and piped loggers for error logs and access logs

x-ref:
"RFE: de-dup file and piped loggers"
https://redmine.lighttpd.net/issu

[multiple] de-dup file and piped loggers (fixes #3101)

de-dup file and piped loggers for error logs and access logs

x-ref:
"RFE: de-dup file and piped loggers"
https://redmine.lighttpd.net/issues/3101

show more ...

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and Fr

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
$ lighttpd -V
+ Y2038 support # Y2038-SAFE
$ lighttpd -V
- Y2038 support (unsafe 32-bit signed time_t) # Y2038-UNSAFE

* Y2038: general platform info
* Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
- all major 64-bit platforms (known to this author) use 64-bit time_t
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
- Linux x32 ABI (different from i686)
- FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
- NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
- OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
- Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
- MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
- e.g. OpenVMS (unknown if lighttpd builds on this platform)
* Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
- Linux 32-bit (including i686)
- glibc 32-bit library support not yet available for 64-bit time_t
- https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
- Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
- https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
"Note: at this point, 64-bit time support in dual-time
configurations is work-in-progress, so for these
configurations, the public API only makes the 32-bit time
support available. In a later change, the public API will
allow user code to choose the time size for a given
compilation unit."
- compiling with -D_TIME_BITS=64 currently has no effect
- glibc recent (Jul 2021) mailing list discussion
- https://public-inbox.org/bug-gnulib/[email protected]/T/
- FreeBSD i386
- DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
* lighttpd prefers system monotonic clock instead of realtime clock
in places where realtime clock is not required
* lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
* (lighttpd presumes that lighttpd will not encounter dates before 1970
during normal operation.)
* lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
to convert negative timestamps for comparisions with 64-bit timestamps
(treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
* lighttpd provides unix_time64_t (int64_t) and
* lighttpd provides struct unix_timespec64 (unix_timespec64_t)
(struct timespec equivalent using unix_time64_t tv_sec member)
* lighttpd provides gmtime64_r() and localtime64_r() wrappers
for platforms 32-bit platforms using 32-bit time_t and
lighttpd temporarily shifts the year in order to use
gmtime_r() and localtime_r() (or gmtime() and localtime())
from standard libraries, before readjusting year and passing
struct tm to formatting functions such as strftime()
* lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
on 32-bit platforms using 32-bit signed time_t, underlying libraries and
underlying filesystems might not behave properly after 32-bit signed time_t
overflows (19 Jan 2038 03:14:08 GMT). If a given 32-bit OS does not work
properly using negative time_t values, then lighttpd likely will not work
properly on that system.

* Other references and blogs
- https://en.wikipedia.org/wiki/Year_2038_problem
- https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
- http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/

show more ...

[core] reduce memcmp in http_request_parse_header

extend http_header_parse_ctx to support enum http_header_h2_e
HTTP/2 pseudo-headers in hpctx->id before using memcmp()

[core] request_config_reset()

move configfile.c:config_reset_config()
to reqpool.c:request_config_reset()

[multiple] static file optimization; reuse cache

reuse cache lookup in common case of serving a static file
rather than repeating the stat_cache_entry lookup
(which is more work than memcmp() to re-

[multiple] static file optimization; reuse cache

reuse cache lookup in common case of serving a static file
rather than repeating the stat_cache_entry lookup
(which is more work than memcmp() to re-check stat_cache_entry match)

show more ...

[core] consistent inclusion of sys-time.h

[mod_auth] close HTTP/2 connection after bad pass

mitigation slows down brute force password attacks

x-ref:
"Possible feature: authentication brute force hardening"
https://redmine.lighttpd.net

[mod_auth] close HTTP/2 connection after bad pass

mitigation slows down brute force password attacks

x-ref:
"Possible feature: authentication brute force hardening"
https://redmine.lighttpd.net/boards/3/topics/8885

show more ...

[core] track Content-Length from backend (fixes #3046)

track Content-Length from backend in r->resp_body_scratchpad

x-ref:
"Failure on second request in http proxy backend"
https://redmine.ligh

[core] track Content-Length from backend (fixes #3046)

track Content-Length from backend in r->resp_body_scratchpad

x-ref:
"Failure on second request in http proxy backend"
https://redmine.lighttpd.net/issues/3046

show more ...

[core] li_restricted_strtoint64()

(make public for reuse)

[core] remove unused r->content_length

(was parsed Content-Length sent by backend)
(might add back in future along with stricter validation of field value)

[multiple] inline chunkqueue where always alloc'd

inline struct chunkqueue where always allocated in other structs

(memory locality)

[core] reuse r->start_hp.tv_sec for r->start_ts

(remove duplicated field from (request_st *))

[core] http_header_e <=> lshpack_static_hdr_idx

map enum http_header_e to enum lshpack_static_hdr_idx
map enum lshpack_static_hdr_idx to enum http_header_e

[multiple] extend enum http_header_e list

[core] http_request_parse_header() specialized

http_request_parse_header() specialized for HTTP/2 request headers
to be parsed as each field-name and value is HPACK-decoded; send headers
directly fr

[core] http_request_parse_header() specialized

http_request_parse_header() specialized for HTTP/2 request headers
to be parsed as each field-name and value is HPACK-decoded; send headers
directly from HPACK decoder, rather than double-buffering in chunkqueue

http_request_headers_process_h2() for post-processing

show more ...