[multiple] employ ck_realloc_u32() shared code

employ ck_realloc_u32() shared code to slightly reduce code size

[core] remove unused srv->con_* counters

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_i

[multiple] simplify bytes_in/bytes_out accounting

encapsulate accounting calculations in
http_request_stats_bytes_in()
http_request_stats_bytes_out()

more accurate accounting for HTTP/1.1 bytes_in on keep-alive requests
(affects case where client pipelines HTTP/1.1 requests)

remove con->bytes_read and con->bytes_written
(no longer needed since request_st was split from connection struct
and request bytes_read_ckpt and bytes_written_ckpt are maintained
for HTTP/1.x bytes_in and bytes_out accounting. Also, further back,
chunkqueue internal accounting was simplified to maintain bytes_in
and bytes_out to always match chunkqueue length)

show more ...

[mod_magnet] lighty.server.* table w/ new function

(experimental)

(new interfaces are unstable and may change)

[mod_cgi] cgi.local-redir request_reset thru fnptr

cgi.local-redir call plugins_request_reset through fn ptr

(isolate plugins_* funcs to server;
should not be called directly from plugins/modules)

[core] (const char *) for srvconf.modules_dir

no need to be allocated (buffer *)

[core] pcre2 support (--with-pcre2)

x-ref:
"lighttpd: depends on obsolete pcre3 library"
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000063

[core] save config regex captures only if used

save config regex captures separately only if used by url.redirect
or url.rewrite replacement directives within the condition
(or for conditions contai

[core] save config regex captures only if used

save config regex captures separately only if used by url.redirect
or url.rewrite replacement directives within the condition
(or for conditions containing directives from any other module
which calls config_capture() for its directives during init)

keep pointer to match data (cond_match_t *) in r->cond_match[]
rather than cond_match_t to reduce data copying in h2_init_stream().
h2_init_stream() copies the results for already-evaluated conditions
to avoid re-evaluating connection-level conditions for each and every
stream. When conditions are reset, then the pointer in r->cond_match[]
is updated when the condition is re-evaluated. (This all assumes that
HTTP/2 connection-level conditions are not unset or re-evaluated once
HTTP/2 streams begin.)

show more ...

[core] allow debug.log-state-handling in condition

allow debug.log-state-handling to be enabled in config conditions,
even though the connection and request may already have begun
(and therefore not

[core] allow debug.log-state-handling in condition

allow debug.log-state-handling to be enabled in config conditions,
even though the connection and request may already have begun
(and therefore not logging state changes which already occurred)

(also, minor additional structure cleanups made in the vicinity)

show more ...

[core] remove HANDLER_WAIT_FOR_FD

If a system call returns EMFILE, then admin should increase
server.max-fds and check/increase rlimits for num files (ulimit -Hn)

Alternatively, the admin might dec

[core] remove HANDLER_WAIT_FOR_FD

If a system call returns EMFILE, then admin should increase
server.max-fds and check/increase rlimits for num files (ulimit -Hn)

Alternatively, the admin might decrease server.max-connections to limit
the number of connections served in parallel.

show more ...

[core] change con joblist to singly-linked-list

avoids separate memory allocation for list of pointers

adds ability to check if con is already in joblist,
so do not re-add con if already in joblist

[core] change con joblist to singly-linked-list

avoids separate memory allocation for list of pointers

adds ability to check if con is already in joblist,
so do not re-add con if already in joblist

since con is checked if in joblist before being added to joblist,
there is no longer need for two lists and jobs can be processed
before poll() for to process new events

show more ...

[core] change srv->conns to doubly-linked-list

avoids separate memory allocation for list of pointers

[core] rename srv->max_conns -> srv->lim_conns

srv->lim_conns tracks remaining conns until limit is reached,
replacing (srv->max_conns - srv->conns.used)

srv->srvconf.max_conns is now updated at st

[core] rename srv->max_conns -> srv->lim_conns

srv->lim_conns tracks remaining conns until limit is reached,
replacing (srv->max_conns - srv->conns.used)

srv->srvconf.max_conns is now updated at startup, so
srv->srvconf.max_conns serves as srv->max_conns

show more ...

[core] conns_pool separate from conns list (#3084)

keep conns_pool of struct connection separate from conns list
and allocate conns list to srv->srvconf.max_conns size at startup

x-ref:
"Memory f

[core] conns_pool separate from conns list (#3084)

keep conns_pool of struct connection separate from conns list
and allocate conns list to srv->srvconf.max_conns size at startup

x-ref:
"Memory fragmentation with HTTP/2 enabled"
https://redmine.lighttpd.net/issues/3084

show more ...

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and Fr

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
$ lighttpd -V
+ Y2038 support # Y2038-SAFE
$ lighttpd -V
- Y2038 support (unsafe 32-bit signed time_t) # Y2038-UNSAFE

* Y2038: general platform info
* Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
- all major 64-bit platforms (known to this author) use 64-bit time_t
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
- Linux x32 ABI (different from i686)
- FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
- NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
- OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
- Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
- MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
- e.g. OpenVMS (unknown if lighttpd builds on this platform)
* Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
- Linux 32-bit (including i686)
- glibc 32-bit library support not yet available for 64-bit time_t
- https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
- Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
- https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
"Note: at this point, 64-bit time support in dual-time
configurations is work-in-progress, so for these
configurations, the public API only makes the 32-bit time
support available. In a later change, the public API will
allow user code to choose the time size for a given
compilation unit."
- compiling with -D_TIME_BITS=64 currently has no effect
- glibc recent (Jul 2021) mailing list discussion
- https://public-inbox.org/bug-gnulib/[email protected]/T/
- FreeBSD i386
- DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
* lighttpd prefers system monotonic clock instead of realtime clock
in places where realtime clock is not required
* lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
* (lighttpd presumes that lighttpd will not encounter dates before 1970
during normal operation.)
* lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
to convert negative timestamps for comparisions with 64-bit timestamps
(treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
* lighttpd provides unix_time64_t (int64_t) and
* lighttpd provides struct unix_timespec64 (unix_timespec64_t)
(struct timespec equivalent using unix_time64_t tv_sec member)
* lighttpd provides gmtime64_r() and localtime64_r() wrappers
for platforms 32-bit platforms using 32-bit time_t and
lighttpd temporarily shifts the year in order to use
gmtime_r() and localtime_r() (or gmtime() and localtime())
from standard libraries, before readjusting year and passing
struct tm to formatting functions such as strftime()
* lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
on 32-bit platforms using 32-bit signed time_t, underlying libraries and
underlying filesystems might not behave properly after 32-bit signed time_t
overflows (19 Jan 2038 03:14:08 GMT). If a given 32-bit OS does not work
properly using negative time_t values, then lighttpd likely will not work
properly on that system.

* Other references and blogs
- https://en.wikipedia.org/wiki/Year_2038_problem
- https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
- http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/

show more ...

[multiple] inline struct in con->dst_addr_buf

(mod_extforward recently changed to use buffer_move() to save addr
instead of swapping pointers)

[multiple] mark con->srv_socket a const ptr

[core] consistent inclusion of sys-time.h

[core] save parsed listen addrs at startup

save parsed listen addrs at startup for reuse at runtime

srv_socket->srv_token is normalized at startup and contains IP and port.
save offset to colon, if

[core] save parsed listen addrs at startup

save parsed listen addrs at startup for reuse at runtime

srv_socket->srv_token is normalized at startup and contains IP and port.
save offset to colon, if present, or else length of string (unix socket)

At runtime, srv_token_colon can be quickly used as length of IP string
(without port) or, if not length of string, offset of stringified port
following the colon.

show more ...

[core] connection_joblist global

con->srv->joblist is expensive to traverse when cache is cold
and called from connection_handle_fdevent(); var made a global

[core] defer handling FDEVENT_HUP and FDEVENT_ERR

defer handling FDEVENT_HUP and FDEVENT_ERR to after processing
(connection *) in order to have a chance to read data in kernel
socket buffers

[core] graceful shutdown timeout option

server.feature-flags += ("server.graceful-shutdown-timeout" => 10)

After receiving SIGINT or SIGUSR1, lighttpd will gracefully shutdown,
waiting for existing

[core] graceful shutdown timeout option

server.feature-flags += ("server.graceful-shutdown-timeout" => 10)

After receiving SIGINT or SIGUSR1, lighttpd will gracefully shutdown,
waiting for existing connections to complete. In the case of SIGUSR1,
this wait occurs before restarting lighttpd. The default timeout is
none (unlimited).

When "server.graceful-shutdown-timeout" option is set, it defines the
number of seconds that lighttpd will wait for existing connections to
complete before shutting down the connection.

Sites which expect large uploads or downloads, or those with very slow
clients, might want to set a much longer timeout, e.g 60 seconds

For more immediate graceful restarts, while still allowing existing
connections time to complete, sites should additionally consider
whether or not
server.feature-flags += ("server.graceful-restart-bg" => "enable")
is appropriate and compatible with their lighttpd.conf settings

show more ...

[core] graceful and immediate restart option

graceful and (nearly) immediate lighttpd restart option

For *some* configurations, it *may* be safe to background the current
lighttpd server (or worker

[core] graceful and immediate restart option

graceful and (nearly) immediate lighttpd restart option

For *some* configurations, it *may* be safe to background the current
lighttpd server (or workers) to continue processing active requests
and, in parallel, to start up a new lighttpd server with a new
configuration. For other configurations, doing so might not be safe!

Therefore, this option must be explicitly configured to enable:
server.feature-flags += ("server.graceful-restart-bg" => "enable")
server.systemd-socket-activation = "enable"

Along with enabling server.feature-flags "server.graceful-restart-bg",
enabling server.systemd-socket-activation allows transfer of open
listening sockets to the new lighttpd server instance, and occurs
without closing the listening sockets and without destroying the
kernel listen backlog queue on the socket.

Safe configurations may include lighttpd.conf which connect to
standalone backend daemons, e.g. proxying to other servers,
including PHP-FPM backends.

Unsafe configurations include lighttpd.conf which use "bin-path" option
in *.server configs, instructing lighttpd to execute the backends.
Using the graceful-and-immediate-restart option is likely *unsafe* if
the backend daemon expects only one instance of itself to run at a time.

Current implementation of graceful and immediate restart option keeps
the backgrounded lighttpd in the same process group, so that subsequent
SIGINT or SIGTERM will shut down both the new and the backgrounded
servers. (An alternative option (commented out in the code) is to
background and detach from the new lighttpd process.) Regardless,
existing subprocesses, such as CGI, remain in original process group.
As a result, the new lighttpd server may receive SIGCHLD for unknown
processes inherited from the old server, which the new lighttpd server
will reap and discard. The original lighttpd server, now a child, will
be unable to detect exit or reap and report status on those pre-existing
subprocesses.

Graceful restart is triggered in lighttpd by sending lighttpd SIGUSR1.
If lighttpd is configured with workers, then SIGINT (not SIGUSR1) is
sent to the process group, including other processes started by
lighttpd, e.g. CGI. To work well with graceful restart, CGI scripts and
other processes should trap SIGINT (and SIGUSR1 for good measure).
Long-running scripts may want to checkpoint and close, e.g. a CGI script
implementing a long-running websocket connection.

show more ...

[core] redirect to dir using relative-path

Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location

Prefer relative path in redirection for the benefit of reve

[core] redirect to dir using relative-path

Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location

Prefer relative path in redirection for the benefit of reverse proxies
and CDNs. Doing so also avoids potentially disclosing internal schemes
and server names which client might not be able to directly reach.

To restore prior behavior of sending a fully-qualified absolute URI:
server.feature-flags += ("absolute-dir-redirect" => "enable")

x-ref:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63357

show more ...

[core] alternate between two joblists

process one joblist and queue work up on second joblist

loop to poll, but poll immediate if work was queued on second joblist