1Multithreading support in memcached 2 3OVERVIEW 4 5By default, memcached is compiled as a single-threaded application. This is 6the most CPU-efficient mode of operation, and it is appropriate for memcached 7instances running on single-processor servers or whose request volume is 8low enough that available CPU power is not a bottleneck. 9 10More heavily-used memcached instances can benefit from multithreaded mode. 11To enable it, use the "--enable-threads" option to the configure script: 12 13./configure --enable-threads 14 15You must have the POSIX thread functions (pthread_*) on your system in order 16to use memcached's multithreaded mode. 17 18Once you have a thread-capable memcached executable, you can control the 19number of threads using the "-t" option; the default is 4. On a machine 20that's dedicated to memcached, you will typically want one thread per 21processor core. Due to memcached's nonblocking architecture, there is no 22real advantage to using more threads than the number of CPUs on the machine; 23doing so will increase lock contention and is likely to degrade performance. 24 25 26INTERNALS 27 28The threading support is mostly implemented as a series of wrapper functions 29that protect calls to underlying code with one of a small number of locks. 30In single-threaded mode, the wrappers are replaced with direct invocations 31of the target code using #define; that is done in memcached.h. This approach 32allows memcached to be compiled in either single- or multi-threaded mode. 33 34Each thread has its own instance of libevent ("base" in libevent terminology). 35The only direct interaction between threads is for new connections. One of 36the threads handles the TCP listen socket; each new connection is passed to 37a different thread on a round-robin basis. After that, each thread operates 38on its set of connections as if it were running in single-threaded mode, 39using libevent to manage nonblocking I/O as usual. 40 41UDP requests are a bit different, since there is only one UDP socket that's 42shared by all clients. The UDP socket is monitored by all of the threads. 43When a datagram comes in, all the threads that aren't already processing 44another request will receive "socket readable" callbacks from libevent. 45Only one thread will successfully read the request; the others will go back 46to sleep or, in the case of a very busy server, will read whatever other 47UDP requests are waiting in the socket buffer. Note that in the case of 48moderately busy servers, this results in increased CPU consumption since 49threads will constantly wake up and find no input waiting for them. But 50short of much more major surgery on the I/O code, this is not easy to avoid. 51 52 53TO DO 54 55The locking is currently very coarse-grained. There is, for example, one 56lock that protects all the calls to the hashtable-related functions. Since 57memcached spends much of its CPU time on command parsing and response 58assembly, rather than managing the hashtable per se, this is not a huge 59bottleneck for small numbers of processors. However, the locking will likely 60have to be refined in the event that memcached needs to run well on 61massively-parallel machines. 62 63One cheap optimization to reduce contention on that lock: move the hash value 64computation so it occurs before the lock is obtained whenever possible. 65Right now the hash is performed at the lowest levels of the functions in 66assoc.c. If instead it was computed in memcached.c, then passed along with 67the key and length into the items.c code and down into assoc.c, that would 68reduce the amount of time each thread needs to keep the hashtable lock held. 69