1======================== 2Performance Improvements 3======================== 4 5------------ 6Module: core 7------------ 8 9:Author: Jan Kneschke 10:Date: $Date: 2004/11/03 22:26:05 $ 11:Revision: $Revision: 1.3 $ 12 13:abstract: 14 handling performance issues in lighttpd 15 16.. meta:: 17 :keywords: lighttpd, performance 18 19.. contents:: Table of Contents 20 21Description 22=========== 23 24Performance Issues 25------------------ 26 27lighttpd is optimized into varying directions. The most important direction is 28performance. The operation system has two major facilities to help lighttpd 29a deliver its best performance. 30 31HTTP Keep-Alive 32--------------- 33 34Disabling keep-alive might help your server if you suffer from a large 35number of open file descriptors. 36 37The defaults for the server are: :: 38 39 server.max-keep-alive-requests = 128 40 server.max-keep-alive-idle = 30 41 server.max-read-idle = 60 42 server.max-write-idle = 360 43 44handling 128 keep-alive requests in a row on a single connection, waiting 30 seconds 45before an unused keep-alive connection gets dropped by lighttpd. 46 47If you handle several connections at once under a high load (let's assume 500 connections 48in parallel for 24h) you might run into the out-of-fd problem described below. :: 49 50 server.max-keep-alive-requests = 4 51 server.max-keep-alive-idle = 4 52 53would release the connections earlier and would free file descriptors without a 54detrimental performance loss. 55 56Disabling keep-alive completely is the last resort if you are still short on file descriptors: :: 57 58 server.max-keep-alive-requests = 0 59 60Event Handlers 61-------------- 62 63The first one is the Event Handler which takes care of notifying the server 64that one of the connections is ready to send or receive. As you can see, 65every OS has at least the select() call which has some limitations. 66 67============ ========== =============== 68OS Method Config Value 69============ ========== =============== 70all select select 71Unix poll poll 72Linux 2.6+ epoll linux-sysepoll 73Solaris /dev/poll solaris-devpoll 74FreeBSD, ... kqueue freebsd-kqueue 75============ ========== =============== 76 77 78For more information on this topic take a look at http://www.kegel.com/c10k.html 79 80Configuration 81````````````` 82 83The event handler can be set by specifying the 'Config Value' from above 84in the ``server.event-handler`` variable 85 86e.g.: :: 87 88 server.event-handler = "linux-sysepoll" 89 90Network Handlers 91---------------- 92 93The basic network interface for all platforms at the syscalls read() and 94write(). Every modern OS provides its own syscall to help network servers 95transfer files as fast as possible. 96 97If you want to send out a file from the webserver, it doesn't make any sense 98to copy the file into the webserver just to write() it back into a socket 99in the next step. 100 101sendfile() minimizes the work in the application and pushes a file directly 102into the network card (ideally). 103 104lighttpd supports all major platform-specific calls: 105 106========== ========== 107OS Method 108========== ========== 109all write 110Unix writev 111Linux 2.4+ sendfile 112Linux 2.6+ sendfile64 113Solaris sendfilev 114FreeBSD sendfile 115========== ========== 116 117The best backend is selected at compile time. In case you want to use 118another backend set: :: 119 120 server.network-backend = "writev" 121 122You can find more information about network backend in: 123 124 https://blog.lighttpd.net/articles/2005/11/11/optimizing-lighty-for-high-concurrent-large-file-downloads 125 126 127Max Connections 128--------------- 129 130As lighttpd is a single-threaded server, its main resource limit is the 131number of file descriptors, which is set to 1024 by default (on most systems). 132 133If you are running a high-traffic site you might want to increase this limit 134by setting :: 135 136 server.max-fds = 2048 137 138This only works if lighttpd is started as root. 139 140Out-of-fd condition 141------------------- 142 143Since file descriptors are used for TCP/IP sockets, files and directories, 144a simple request for a PHP page might result in using 3 file descriptors: 145 1461. the TCP/IP socket to the client 1472. the TCP/IP and Unix domain socket to the FastCGI process 1483. the filehandle to the file in the document root to check if it exists 149 150If lighttpd runs out of file descriptors, it will stop accepting new 151connections for awhile to use the existing file descriptors to handle the 152currently-running requests. 153 154If more than 90% of the file descriptors are used then the handling of new 155connections is disabled. If it drops below 80% again new connections will 156be accepted again. 157 158Under some circumstances you will see :: 159 160 ... accept() failed: Too many open files 161 162in the error log. This tells you there were too many new requests at once 163and lighttpd could not disable the incoming connections soon enough. The 164connection was dropped and the client received an error message like 'connection 165failed'. This is very rare and might only occur in test setups. 166 167Increasing the ``server.max-fds`` limit will reduce the probability of this 168problem. 169 170stat() cache 171============ 172 173A stat(2) can be expensive; caching it saves time and context switches. 174 175Instead of using stat() every time to check for the existence of a file 176you can stat() it once and monitor the directory the file is in for 177modifications. As long as the directory doesn't change, the files in it 178must all still be the same. 179 180With the help of FAM or gamin you can use kernel events to assure that 181your stat cache is up to date. :: 182 183 server.stat-cache-engine = "fam" # either fam, simple or disabled 184 185 186Platform-Specific Notes 187======================= 188 189Linux 190----- 191 192For Linux 2.4.x you should think about compiling lighttpd with the option 193``--disable-lfs`` to disable the support for files larger than 2GB. lighttpd will 194fall back to the ``writev() + mmap()`` network calls which is ok, but not as 195fast as possible but support files larger than 2GB. 196 197Disabling the TCP options reduces the overhead of each TCP packet and might 198help to get the last few percent of performance out of the server. Be aware that 199disabling these options most likely decreases performance for high-latency and lossy 200links. 201 202- net.ipv4.tcp_sack = 0 203- net.ipv4.tcp_timestamps = 0 204 205Increasing the TCP send and receive buffers will increase the performance a 206lot if (and only if) you have a lot of large files to send. 207 208- net.ipv4.tcp_wmem = 4096 65536 524288 209- net.core.wmem_max = 1048576 210 211If you have a lot of large file uploads, increasing the receive buffers will help. 212 213- net.ipv4.tcp_rmem = 4096 87380 524288 214- net.core.rmem_max = 1048576 215 216Keep in mind that every TCP connection uses the configured amount of memory for socket 217buffers. If you've got many connections this can quickly drain the available memory. 218 219See http://www.acc.umu.se/~maswan/linux-netperf.txt for more information on these parameters. 220 221FreeBSD 222------- 223 224On FreeBSD you might gain some performance by enabling accept filters. Just 225compile your kernel with: :: 226 227 options ACCEPT_FILTER_HTTP 228 229For more ideas about tuning FreeBSD read: tuning(7) 230 231Reducing the recvspace should always be ok if the server only handles HTTP 232requests without large uploads. Increasing the sendspace would reduce the 233system load if you have a lot of large files to be sent, but keep in mind that 234you have to provide the memory in the kernel for each connection. 1024 * 64KB 235would mean 64MB of kernel RAM. Keep this in mind. 236 237- net.inet.tcp.recvspace = 4096 238 239