[core] add more const to stat_cache_update_entry()

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and Fr

[multiple] Y2038 32-bit signed time_t mitigations

Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
$ lighttpd -V
+ Y2038 support # Y2038-SAFE
$ lighttpd -V
- Y2038 support (unsafe 32-bit signed time_t) # Y2038-UNSAFE

* Y2038: general platform info
* Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
- all major 64-bit platforms (known to this author) use 64-bit time_t
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
- Linux x32 ABI (different from i686)
- FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
- NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
- OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
- Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
- MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
* Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
- e.g. OpenVMS (unknown if lighttpd builds on this platform)
* Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
- Linux 32-bit (including i686)
- glibc 32-bit library support not yet available for 64-bit time_t
- https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
- Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
- https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
"Note: at this point, 64-bit time support in dual-time
configurations is work-in-progress, so for these
configurations, the public API only makes the 32-bit time
support available. In a later change, the public API will
allow user code to choose the time size for a given
compilation unit."
- compiling with -D_TIME_BITS=64 currently has no effect
- glibc recent (Jul 2021) mailing list discussion
- https://public-inbox.org/bug-gnulib/[email protected]/T/
- FreeBSD i386
- DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
* lighttpd prefers system monotonic clock instead of realtime clock
in places where realtime clock is not required
* lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
* (lighttpd presumes that lighttpd will not encounter dates before 1970
during normal operation.)
* lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
to convert negative timestamps for comparisions with 64-bit timestamps
(treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
* lighttpd provides unix_time64_t (int64_t) and
* lighttpd provides struct unix_timespec64 (unix_timespec64_t)
(struct timespec equivalent using unix_time64_t tv_sec member)
* lighttpd provides gmtime64_r() and localtime64_r() wrappers
for platforms 32-bit platforms using 32-bit time_t and
lighttpd temporarily shifts the year in order to use
gmtime_r() and localtime_r() (or gmtime() and localtime())
from standard libraries, before readjusting year and passing
struct tm to formatting functions such as strftime()
* lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
on 32-bit platforms using 32-bit signed time_t, underlying libraries and
underlying filesystems might not behave properly after 32-bit signed time_t
overflows (19 Jan 2038 03:14:08 GMT). If a given 32-bit OS does not work
properly using negative time_t values, then lighttpd likely will not work
properly on that system.

* Other references and blogs
- https://en.wikipedia.org/wiki/Year_2038_problem
- https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
- http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/

show more ...

[core] consistent inclusion of sys-time.h

[multiple] add attrs from gcc -Wsuggest-attribute=

[core] use kqueue() instead of FAM/gamin on *BSD

Note: there have always been limitations with lighttpd stat_cache.[ch]
using FAM/gamin on *BSD via kqueue() as lighttpd stat_cache.[ch] only
monitors

[core] use kqueue() instead of FAM/gamin on *BSD

Note: there have always been limitations with lighttpd stat_cache.[ch]
using FAM/gamin on *BSD via kqueue() as lighttpd stat_cache.[ch] only
monitors directories. This kqueue() implementation also only monitors
directories and has limitations.

lighttpd stat_cache.[ch] is notified about additions and removals of
files within a monitored directory but might not be notified of changes
such as timestamps (touch), ownership, or even changes in contents
(e.g. if a file is edited through a hard link)

server.stat-cache-engine = "disable" should be used when files should
not be cached. Full stop. Similarly, "disable" is recommended if files
change frequently. If using server.stat-cache-engine with any engine,
there are caching effects and tradeoffs.

On *BSD and using kqueue() on directories, any change detected clears
the stat_cache of all entries in that directory, since monitoring only
the directory does not indicate which file was added or removed. This
is not efficient for directories containing frequently changed files.

show more ...

[core] use inotify in stat_cache.[ch] on Linux

use inotify in stat_cache.[ch] on Linux, replacing FAM/gamin

[core] stat_cache_entry reference counting

future: should probably create fd cache separate from stat_cache,
perhaps along w/ http-specific fields like etag and content_type
and mayb

[core] stat_cache_entry reference counting

future: should probably create fd cache separate from stat_cache,
perhaps along w/ http-specific fields like etag and content_type
and maybe even mmap

show more ...

[multiple] stat_cache_path_stat() for struct st

stat_cache_path_stat() for cached (struct st *)

[core] stat_cache_get_entry_open()

simple interface to cache open file by extending struct stat_cache_entry

future: should probably create fd cache separate from stat_cache,
perhaps along w

[core] stat_cache_get_entry_open()

simple interface to cache open file by extending struct stat_cache_entry

future: should probably create fd cache separate from stat_cache,
perhaps along w/ http-specific fields like etag and content_type

show more ...

[core] stat_cache_path_isdir()

[core] perf adjustments to avoid load miss

[core] uint32_t is plenty large for path names

[multiple] split con, request (very large change)

NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
chun

[multiple] split con, request (very large change)

NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().

show more ...

[core] isolate stat_cache subsystem

stat_cache.c no longer directly uses struct server *srv

[multiple] stat_cache singleton

[multiple] connection hooks no longer get (srv *)

(explicit (server *) not passed; available in con->srv)

[core] create http chunk header on the stack

streamline code in http_chunk.c

[core] stat_cache_path_contains_symlink use errh

use log_error() with con->errh

[core] use config_plugin_values_init()

[mod_webdav] invalidate parent dir in stat_cache

invalidate directory in stat_cache when a new file or dir is created
within that directory

[stat_cache] FAM: improve monitoring, cache 16 sec

improve FAM monitoring to use reference counting in internal cache

revalidate entries upon use after 16 sec to recover from missed events

(see co

[stat_cache] FAM: improve monitoring, cache 16 sec

improve FAM monitoring to use reference counting in internal cache

revalidate entries upon use after 16 sec to recover from missed events

(see comments in commit for more details about limitations)

show more ...

[stat_cache] interfaces to invalidate entries

[stat_cache] separate func for symlink policy chk

Note: historical ToC-ToU race condition still exists in implementation
server.follow-symlink = "disable" is not recommended (default: "enable")

[core] pass conf.follow_symlink in more places

[core] mark startup/shutdown funcs cold