[multiple] employ ck_calloc, ck_malloc shared code

employ ck_calloc(), ck_malloc() shared code to slightly reduce code size
(centralize the ck_assert() to check that memory allocation succeeded)

[multiple] mark mod_*_plugin_init() funcs cold

[multiple] rename status_counter -> plugin_stats

[mod_ajp13,mod_fastcgi] recv_parse smaller funcs

break *_recv_parse() into a pair of slightly smaller funcs

[mod_ajp13,mod_fastcgi] check resp w/ content len

limit response body from mod_ajp13 and mod_fastcgi to Content-Length,
if Content-Length is provided in response headers; discard excess

[mod_ajp13,mod_fastcgi] comment: no response body

add comment for handling of streaming with no response body

add commented-out code to disable streaming to wait for backend protocol
to signal end

[mod_ajp13,mod_fastcgi] comment: no response body

add comment for handling of streaming with no response body

add commented-out code to disable streaming to wait for backend protocol
to signal end of response (prevent http_response_write_prepare() from
short-circuiting and finishing responses without response body)

x-ref:
"FastCGI premature socket close with response streaming and 204 status"
https://redmine.lighttpd.net/boards/2/topics/10066

show more ...

[core] disable streaming response with authorizer (fixes #3106)

disable streaming response while processing "authorizer" mode
until "authorizer" response 200 OK from the backend is complete

(thx je

[core] disable streaming response with authorizer (fixes #3106)

disable streaming response while processing "authorizer" mode
until "authorizer" response 200 OK from the backend is complete

(thx jefftharris)

x-ref:
"FastCGI authorizer hang with server.stream-response-body"
https://redmine.lighttpd.net/boards/2/topics/9969
"FastCGI authorizer hang with server.stream-response-body"
https://redmine.lighttpd.net/issues/3106

show more ...

[multiple] internal control for backend read bytes

separate internal control for backend max_per_read

When not streaming, large reads will be flushed to temp files on disk.
When streaming, use a sm

[multiple] internal control for backend read bytes

separate internal control for backend max_per_read

When not streaming, large reads will be flushed to temp files on disk.
When streaming, use a smaller buffer to help reduce memory usage.

When not streaming, attempt to read and empty kernel socket bufs.
(e.g. MAX_READ_LIMIT 256k)

When writing to sockets (or pipes) attempt to fill kernel socket bufs.
(e.g. MAX_WRITE_LIMIT 256k)

show more ...

[core] log_error_multiline()

rename log_error_multiline_buffer() to log_error_multiline()
and take (char *)ptr and (size_t)len instead of (buffer *)b

When debug printing request and response header

[core] log_error_multiline()

rename log_error_multiline_buffer() to log_error_multiline()
and take (char *)ptr and (size_t)len instead of (buffer *)b

When debug printing request and response headers,
print each header on separate line for readability
and omit '\r' if "\r\n" ends line

show more ...

[multiple] remove base.h include where not used

(substitute request.h if file only accesses request_st,
and not connection or server structs)

[multiple] reduce redundant NULL buffer checks

This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot co

[multiple] reduce redundant NULL buffer checks

This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.

Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.

In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func. In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.

- check for empty strings at config time and set value to NULL if blank
string will be ignored at runtime; at runtime, simple pointer check
for NULL can be used to check for a value that has been set and is not
blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
and use buffer_is_unset() instead of buffer_is_empty(),
where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
truncate string, and use buffer_extend() to extend

Examples where buffer known not to be NULL:
- cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
(though we might set it to NULL if buffer_is_blank(cpv->v.b))
- address of buffer is arg (&foo)
(compiler optimizer detects this in most, but not all, cases)
- buffer is checked for NULL earlier in func
- buffer is accessed in same scope without a NULL check (e.g. b->ptr)

internal behavior change:
callers must not pass a NULL buffer to some funcs.
- buffer_init_buffer() requires non-null args
- buffer_copy_buffer() requires non-null args
- buffer_append_string_buffer() requires non-null args
- buffer_string_space() requires non-null arg

show more ...

[multiple] use buffer_append_* aggregates

reduces the number of round-trips into some frequently-called routines

[multiple] optimize primitives, buffer_extend()

optimize buffer_* primitives

Other than buffer_string_set_length(), reallocate with one power-2 step
in size (or use the requested size, if larger).

[multiple] optimize primitives, buffer_extend()

optimize buffer_* primitives

Other than buffer_string_set_length(), reallocate with one power-2 step
in size (or use the requested size, if larger). This replaces the fixed
BUFFER_PIECE_SIZE round-up of only 64 bytes extension each reallocation,
which could lead to excessive reallocations in some scenarios.

buffer_extend() convenience routine to prep for batch append
(combines buffer_string_prepare_append() and buffer_commit())

mod_fastcgi, mod_scgi, mod_proxy and others now leverage buffer_extend()

mod_scgi directly performs little-endian encoding of short ints

http_response_write_header() optimizes writing response header,
leveraging buffer_extend()

modify mod_proxy to append line ends
similar to how it is done in http_response_write_header()
(removes one call to buffer_append_string_len())

show more ...

[core] http_cgi.[ch] CGI interfaces (RFC 3875)

collect Common Gateway Interface (CGI) interfaces (RFC 3875)

[multiple] quiet some clang-analyzer warnings

[tests] remove FastCGI test dependency on libfcgi

- rewrite fcgi-responder as standalone app
fcgi-responder is now a minimal, standalone FastCGI server for tests
- remove dependency on fcgi-devel

[tests] remove FastCGI test dependency on libfcgi

- rewrite fcgi-responder as standalone app
fcgi-responder is now a minimal, standalone FastCGI server for tests
- remove dependency on fcgi-devel package
- merge fcgi-auth into fcgi-responder

show more ...

[mod_fastcgi] move src/fastcgi.h into src/compat/

[multiple] add back-pressure gw data pump (fixes #3033)

When server.stream-request-body = 0 (the default), the entire request
body is collected before engaging the backend. For backends which
requi

[multiple] add back-pressure gw data pump (fixes #3033)

When server.stream-request-body = 0 (the default), the entire request
body is collected before engaging the backend. For backends which
require data framing, this could lead to growth in memory use as large
requests were framed all at once.

Prefer to retain large request bodies in temporary files on disk and
frame in portions as write queue to backend drains below a threshold.

x-ref:
"Memory Growth with PUT and full buffered streams"
https://redmine.lighttpd.net/issues/3033

show more ...

[multiple] handle NULL val as empty in *_env_add (fixes #3030)

(bug on master branch; never released)

(thx flynn)

x-ref:
"Fastcgi fails if server.tag is empty"
https://redmine.lighttpd.net/iss

[multiple] handle NULL val as empty in *_env_add (fixes #3030)

(bug on master branch; never released)

(thx flynn)

x-ref:
"Fastcgi fails if server.tag is empty"
https://redmine.lighttpd.net/issues/3030

show more ...

[mod_fastcgi] decode chunked is cold code path

decode chunked from FastCGI backend is cold code path

[multiple] inline chunkqueue where always alloc'd

inline struct chunkqueue where always allocated in other structs

(memory locality)

[multiple] code reuse chunkqueue_peek_data()

code reuse of chunkqueue_peek_data() and chunkqueue_read_data()

[multiple] inline chunkqueue_length()

[mod_fastcgi] decode Transfer-Encoding: chunked

decode Transfer-Encoding: chunked from mod_fastcgi backend

[multiple] rename connection_reset hook to request

rename connection_reset to handle_request_reset