1======================== 2Performance Improvements 3======================== 4 5------------ 6Module: core 7------------ 8 9:Author: Jan Kneschke 10:Date: $Date: 2004/11/03 22:26:05 $ 11:Revision: $Revision: 1.3 $ 12 13:abstract: 14 handling performance issues in lighttpd 15 16.. meta:: 17 :keywords: lighttpd, performance 18 19.. contents:: Table of Contents 20 21Description 22=========== 23 24Performance Issues 25------------------ 26 27lighttpd is optimized into varying directions. The most important direction is 28performance. The operation system has two major facilities to help lighttpd 29a deliver its best performance. 30 31HTTP Keep-Alive 32--------------- 33 34Disabling keep-alive might help your server if you suffer from a large 35number of open file descriptors. 36 37The defaults for the server are: :: 38 39 server.max-keep-alive-requests = 128 40 server.max-keep-alive-idle = 30 41 server.max-read-idle = 60 42 server.max-write-idle = 360 43 44handling 128 keep-alive requests in a row on a single connection, waiting 30 seconds 45before an unused keep-alive connection gets dropped by lighttpd. 46 47If you handle several connections at once under a high load (let's assume 500 connections 48in parallel for 24h) you might run into the out-of-fd problem described below. :: 49 50 server.max-keep-alive-requests = 4 51 server.max-keep-alive-idle = 4 52 53would release the connections earlier and would free file descriptors without a 54detrimental performance loss. 55 56Disabling keep-alive completely is the last resort if you are still short on file descriptors: :: 57 58 server.max-keep-alive-requests = 0 59 60Event Handlers 61-------------- 62 63The first one is the Event Handler which takes care of notifying the server 64that one of the connections is ready to send or receive. As you can see, 65every OS has at least the select() call which has some limitations. 66 67============ ========== =============== 68OS Method Config Value 69============ ========== =============== 70all select select 71Unix poll poll 72Linux 2.4+ rt-signals linux-rtsig 73Linux 2.6+ epoll linux-sysepoll 74Solaris /dev/poll solaris-devpoll 75FreeBSD, ... kqueue freebsd-kqueue 76============ ========== =============== 77 78 79For more information on this topic take a look at http://www.kegel.com/c10k.html 80 81Configuration 82````````````` 83 84The event handler can be set by specifying the 'Config Value' from above 85in the ``server.event-handler`` variable 86 87e.g.: :: 88 89 server.event-handler = "linux-sysepoll" 90 91Network Handlers 92---------------- 93 94The basic network interface for all platforms at the syscalls read() and 95write(). Every modern OS provides its own syscall to help network servers 96transfer files as fast as possible. 97 98If you want to send out a file from the webserver, it doesn't make any sense 99to copy the file into the webserver just to write() it back into a socket 100in the next step. 101 102sendfile() minimizes the work in the application and pushes a file directly 103into the network card (ideally). 104 105lighttpd supports all major platform-specific calls: 106 107========== ========== 108OS Method 109========== ========== 110all write 111Unix writev 112Linux 2.4+ sendfile 113Linux 2.6+ sendfile64 114Solaris sendfilev 115FreeBSD sendfile 116========== ========== 117 118The best backend is selected at compile time. In case you want to use 119another backend set: :: 120 121 server.network-backend = "writev" 122 123You can find more information about network backend in: 124 125 http://blog.lighttpd.net/articles/2005/11/11/optimizing-lighty-for-high-concurrent-large-file-downloads 126 127 128Max Connections 129--------------- 130 131As lighttpd is a single-threaded server, its main resource limit is the 132number of file descriptors, which is set to 1024 by default (on most systems). 133 134If you are running a high-traffic site you might want to increase this limit 135by setting :: 136 137 server.max-fds = 2048 138 139This only works if lighttpd is started as root. 140 141Out-of-fd condition 142------------------- 143 144Since file descriptors are used for TCP/IP sockets, files and directories, 145a simple request for a PHP page might result in using 3 file descriptors: 146 1471. the TCP/IP socket to the client 1482. the TCP/IP and Unix domain socket to the FastCGI process 1493. the filehandle to the file in the document root to check if it exists 150 151If lighttpd runs out of file descriptors, it will stop accepting new 152connections for awhile to use the existing file descriptors to handle the 153currently-running requests. 154 155If more than 90% of the file descriptors are used then the handling of new 156connections is disabled. If it drops below 80% again new connections will 157be accepted again. 158 159Under some circumstances you will see :: 160 161 ... accept() failed: Too many open files 162 163in the error log. This tells you there were too many new requests at once 164and lighttpd could not disable the incoming connections soon enough. The 165connection was dropped and the client received an error message like 'connection 166failed'. This is very rare and might only occur in test setups. 167 168Increasing the ``server.max-fds`` limit will reduce the probability of this 169problem. 170 171stat() cache 172============ 173 174A stat(2) can be expensive; caching it saves time and context switches. 175 176Instead of using stat() every time to check for the existence of a file 177you can stat() it once and monitor the directory the file is in for 178modifications. As long as the directory doesn't change, the files in it 179must all still be the same. 180 181With the help of FAM or gamin you can use kernel events to assure that 182your stat cache is up to date. :: 183 184 server.stat-cache-engine = "fam" # either fam, simple or disabled 185 186 187Platform-Specific Notes 188======================= 189 190Linux 191----- 192 193For Linux 2.4.x you should think about compiling lighttpd with the option 194``--disable-lfs`` to disable the support for files larger than 2GB. lighttpd will 195fall back to the ``writev() + mmap()`` network calls which is ok, but not as 196fast as possible but support files larger than 2GB. 197 198Disabling the TCP options reduces the overhead of each TCP packet and might 199help to get the last few percent of performance out of the server. Be aware that 200disabling these options most likely decreases performance for high-latency and lossy 201links. 202 203- net.ipv4.tcp_sack = 0 204- net.ipv4.tcp_timestamps = 0 205 206Increasing the TCP send and receive buffers will increase the performance a 207lot if (and only if) you have a lot of large files to send. 208 209- net.ipv4.tcp_wmem = 4096 65536 524288 210- net.core.wmem_max = 1048576 211 212If you have a lot of large file uploads, increasing the receive buffers will help. 213 214- net.ipv4.tcp_rmem = 4096 87380 524288 215- net.core.rmem_max = 1048576 216 217Keep in mind that every TCP connection uses the configured amount of memory for socket 218buffers. If you've got many connections this can quickly drain the available memory. 219 220See http://www.acc.umu.se/~maswan/linux-netperf.txt for more information on these parameters. 221 222FreeBSD 223------- 224 225On FreeBSD you might gain some performance by enabling accept filters. Just 226compile your kernel with: :: 227 228 options ACCEPT_FILTER_HTTP 229 230For more ideas about tuning FreeBSD read: tuning(7) 231 232Reducing the recvspace should always be ok if the server only handles HTTP 233requests without large uploads. Increasing the sendspace would reduce the 234system load if you have a lot of large files to be sent, but keep in mind that 235you have to provide the memory in the kernel for each connection. 1024 * 64KB 236would mean 64MB of kernel RAM. Keep this in mind. 237 238- net.inet.tcp.recvspace = 4096 239 240