[core] chunk.c tweaks

add branch hint to chunkqueue_use_memory()

remove an overly paranoid check for chunkqueue corruption
(were it needed, it would need to be applied in numerous other places)

[multiple] employ ck_calloc, ck_malloc shared code

employ ck_calloc(), ck_malloc() shared code to slightly reduce code size
(centralize the ck_assert() to check that memory allocation succeeded)

[core] handle ENOSPC with pwritev() (#3171)

handle ENOSPC with pwritev() and fallback to next item in
server.upload-dirs list

x-ref:
"File upload regression with --disable-lfs"
https://redmine.

[core] handle ENOSPC with pwritev() (#3171)

handle ENOSPC with pwritev() and fallback to next item in
server.upload-dirs list

x-ref:
"File upload regression with --disable-lfs"
https://redmine.lighttpd.net/issues/3171

show more ...

[core] chunkqueue_steal() handle unexpected 0 len

pedantic: chunkqueue_steal() handle unexpected 0 len and 0-length chunks

more code reuse (slightly more instructions) when pwritev not available

[core] chunk_file_pread() to wrap pread()

[multiple] shared code for struct chunk and mmap

chunkqueue_chunk_file_view()

reduces size of struct chunk

use mmap with mod_deflate libdeflate, if mmap available,
even when lighttpd not built w

[multiple] shared code for struct chunk and mmap

chunkqueue_chunk_file_view()

reduces size of struct chunk

use mmap with mod_deflate libdeflate, if mmap available,
even when lighttpd not built with --enable-mmap

avoid using mmap on temp files in chunkqueue (c->file.is_temp)
since pread() with a reasonable block size is typically as fast
or faster than mmap on files read sequentially and used only once,
especially when writing results to limited-size socket buffers
(and lighttpd temp files are, in most cases, by default about 1 MB)
(Exception: sometimes mmap is used for convenience or to fulfill
a requirement, e.g. one-shot libdeflate in mod_deflate)
(There are many factors which influence speed of mmap versus pread,
so this should not be construed as generic usage advice.)

show more ...

[multiple] pedantic chunk.c checks for 0-len chunk

[multiple] use new sys_setjmp_eval3() interface

wrap system os setjmp or compiler try/catch and access mmap inside
callback to protect against SIGBUS (or _WIN32 EXCEPTION_IN_PAGE_ERROR)

[core] chunkqueue_small_resp_optim() comment

[core] tighten chunkqueue_small_resp_optim()

[core] tighten chunkqueue_mark_written; better asm

chunkqueue_mark_written() also removes finished chunks from beginning of
chunkqueue instead of separate call to chunkqueue_remove_finished_chunks

[core] pwritev w/ chunkqueue_steal_with_tempfiles

use pwritev(), where available, with chunkqueue_steal_with_tempfiles()

The initial write to tempfiles (when response initially grows too
large in m

[core] pwritev w/ chunkqueue_steal_with_tempfiles

use pwritev(), where available, with chunkqueue_steal_with_tempfiles()

The initial write to tempfiles (when response initially grows too
large in memory) can be multiple MEM_CHUNKs, so attempt to use
single pwritev() syscall

show more ...

[core] tighten chunkqueue_steal* code; better asm

[core] remove redundant checks in same context

[core] expose chunkqueue_remove_empty_chunks()

expose chunkqueue_remove_empty_chunks() for use in error recovery
when aborting a chunkqueue_append_buffer_open*()

[multiple] __attribute_nonnull__ now takes params

define __attribute_nonnull__(params) with params to match
recent changes in glibc development (targetting glibc 2.35 in Feb 2022)

x-ref:
new __at

[multiple] __attribute_nonnull__ now takes params

define __attribute_nonnull__(params) with params to match
recent changes in glibc development (targetting glibc 2.35 in Feb 2022)

x-ref:
new __attribute_nonnull__(params) conflicts with third-party
https://sourceware.org/bugzilla/show_bug.cgi?id=28425

show more ...

[core] portability tweaks for various platforms/cc

[core] quiet coverity warnings

[core] chunkqueue_peek_data() mmap experiment

chunkqueue_peek_data() experiment to mmap temporary files

(currently disabled in code due to not making measurable difference
in throughput (during a

[core] chunkqueue_peek_data() mmap experiment

chunkqueue_peek_data() experiment to mmap temporary files

(currently disabled in code due to not making measurable difference
in throughput (during a specific microbenchmark load test),
though it does reduce CPU use by ~10% in the same microbenchmark)

enabling this may cause large spikes in RSS mem usage reported by the
system, due to the read-only memory maps of the temporary files,
but this is nothing to be alarmed about, as the memory maps are
file-backed and read-only, so minimally add to memory pressure

show more ...

[core] chunkqueue_mmap_chunk_len() for code reuse

(pull some code into smaller routines for potential code reuse)

[core] use pread() to skip lseek()

[core] splice() data from backends to tempfiles

splice() data from backends to tempfiles (where splice() is available);
reduce copying data to userspace when writing data to tempfiles

Note: splice(

[core] splice() data from backends to tempfiles

splice() data from backends to tempfiles (where splice() is available);
reduce copying data to userspace when writing data to tempfiles

Note: splice() on Linux returns EINVAL if target file has O_APPEND set
so lighttpd uses pwrite() (where available) when writing to tempfiles
(instead of lseek() + write(), or O_APPEND and write())

show more ...

[multiple] avoid empty chunks in chunkqueue

(This might render some calls to chunkqueue_remove_finished_chunks()
no longer necessary)

[build] check for pread(), pwrite(), splice()

[multiple] fdevent_mkostemp()

fdevent_mkostemp() with flags arg so that caller can pass O_APPEND

renamed from fdevent_mkstemp_append(), previously always O_APPEND