[core] remove some (now) unused http_chunk APIs

remove http_chunk_append_file() and http_chunk_append_file_range()

callers should choose to use stat_cache_entry_open() for caching
or should open fi

[core] remove some (now) unused http_chunk APIs

remove http_chunk_append_file() and http_chunk_append_file_range()

callers should choose to use stat_cache_entry_open() for caching
or should open file and check sizes and ranges

show more ...

[core] http_chunk_append_file_ref_range()

http_chunk_append_file_ref() and http_chunk_append_file_ref_range()
to take stat_cache_entry ref and append FILE_CHUNK

[core] pass open fd to http_response_parse_range

[core] decode Transfer-Encoding: chunked from gw

decode Transfer-Encoding: chunked from gw (gateway backends)

Transfer-Encoding: chunked is a hop-by-hop header.

Handling chunked encoding remove a

[core] decode Transfer-Encoding: chunked from gw

decode Transfer-Encoding: chunked from gw (gateway backends)

Transfer-Encoding: chunked is a hop-by-hop header.

Handling chunked encoding remove a hurdle for mod_proxy to send HTTP/1.1
requests to backends and be able to handle HTTP/1.1 responses.

Other backends ought not to send Transfer-Encoding: chunked, but in
practice, some implementations do.

show more ...

[multiple] split con, request (very large change)

NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
chun

[multiple] split con, request (very large change)

NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().

show more ...

[core] create http chunk header on the stack

streamline code in http_chunk.c

[multiple] prefer (connection *) to (srv *)

convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passi

[multiple] prefer (connection *) to (srv *)

convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)

show more ...

[core] chunkqueue perf: read small files into mem

[core] http_chunk_append_file_fd()

[mod_fastcgi] perf: reduce data copies

http_chunk_transfer_cqlen()

[core] some header cleanup

provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures

[core] some header cleanup

provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers

show more ...

[core] buffer large responses to tempfiles (fixes #758, fixes #760, fixes #933, fixes #1387, #1283, fixes #2083)

This replaces buffering entire response in memory which might lead to
huge memory foo

[core] buffer large responses to tempfiles (fixes #758, fixes #760, fixes #933, fixes #1387, #1283, fixes #2083)

This replaces buffering entire response in memory which might lead to
huge memory footprint and possibly to memory exhaustion.

use tempfiles of fixed size so disk space is freed as each file sent

update callers of http_chunk_append_mem() and http_chunk_append_buffer()
to handle failures when writing to tempfile.

x-ref:
"memory fragmentation leads to high memory usage after peaks"
https://redmine.lighttpd.net/issues/758
"Random crashing on FreeBSD 6.1"
https://redmine.lighttpd.net/issues/760
"lighty should buffer responses (after it grows above certain size) on disk"
https://redmine.lighttpd.net/issues/933
"Memory usage increases when proxy+ssl+large file"
https://redmine.lighttpd.net/issues/1283
"lighttpd+fastcgi memory problem"
https://redmine.lighttpd.net/issues/1387
"Excessive Memory usage with streamed files from PHP"
https://redmine.lighttpd.net/issues/2083

show more ...

[core] open fd when appending file to cq (fixes #2655)

http_chunk_append_file() opens fd when appending file to chunkqueue.
Defers calculation of content length until response is finished.

This red

[core] open fd when appending file to cq (fixes #2655)

http_chunk_append_file() opens fd when appending file to chunkqueue.
Defers calculation of content length until response is finished.

This reduces race conditions pertaining to stat() and then (later)
open(), when the result of the stat() was used for Content-Length
or to generate chunked headers.

Note: this does not change how lighttpd handles files that are modified
in-place by another process after having been opened by lighttpd --
don't do that. This *does* improve handling of files that are
frequently modified via a temporary file and then atomically renamed
into place.

mod_fastcgi has been modified to use http_chunk_append_file_range() with
X-Sendfile2 and will open the target file multiple times if there are
multiple ranges.

Note: (future todo) not implemented for chunk.[ch] interfaces used by
range requests in mod_staticfile or by mod_ssi. Those uses could lead
to too many open fds. For mod_staticfile, limits should be put in place
for max number of ranges accepted by mod_staticfile. For mod_ssi,
limits would need to be placed on the maximum number of includes, and
the primary SSI file split across lots of SSI directives should either
copy the pieces or perhaps chunk.h could be extended to allow for an
open fd to be shared across multiple chunks. Doing either of these
would improve the performance of SSI since they would replace many file
opens on the pieces of the SSI file around the SSI directives.

x-ref:
"Serving a file that is getting updated can cause an empty response or incorrect content-length error"
https://redmine.lighttpd.net/issues/2655

github:
Closes #49

show more ...

consistent inclusion of config.h at top of files (fixes #2073)

From: Glenn Strauss <[email protected]>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3113 152afb58-edef-03

consistent inclusion of config.h at top of files (fixes #2073)

From: Glenn Strauss <[email protected]>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3113 152afb58-edef-0310-8abb-c4023f1b3aa9

show more ...

fix buffer, chunk and http_chunk API

* remove unused structs and functions
(buffer_array, read_buffer)
* change return type from int to void for many functions,
as the return value (indi

fix buffer, chunk and http_chunk API

* remove unused structs and functions
(buffer_array, read_buffer)
* change return type from int to void for many functions,
as the return value (indicating error/success) was never checked,
and the function would only fail on programming errors and not on
invalid input; changed functions to use force_assert instead of
returning an error.
* all "len" parameters now are the real size of the memory to be read.
the length of strings is given always without the terminating 0.
* the "buffer" struct still counts the terminating 0 in ->used,
provide buffer_string_length() to get the length of a string in a
buffer.
unset config "strings" have used == 0, which is used in some places
to distinguish unset values from "" (empty string) values.
* most buffer usages should now use it as string container.
* optimise some buffer copying by "moving" data to other buffers
* use (u)intmax_t for generic int-to-string functions
* remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
* converted BUFFER_APPEND_SLASH to inline function (no macro feature
needed)
* refactor: create chunkqueue_steal: moving (partial) chunks into another
queue
* http_chunk: added separate function to terminate chunked body instead of
magic handling in http_chunk_append_mem().
http_chunk_append_* now handle empty chunks, and never terminate the
chunked body.

From: Stefan Bühler <[email protected]>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2975 152afb58-edef-0310-8abb-c4023f1b3aa9

show more ...

moved everything below trunk/ and added branches/ and tags/


git-svn-id: svn://svn.lighttpd.net/lighttpd/trunk@30 152afb58-edef-0310-8abb-c4023f1b3aa9