1NAPI ID based worker thread selection
2  -N <num_napi_ids>  | --napi_ids=<num_napi_ids>
3
4By default memcached assigns connections to worker threads in a round-robin
5manner. NAPI ID based worker thread selection enables worker threads to be
6selected based on the NIC HW RX queue on which the incoming request is
7received.
8
9This is enabled using SO_INCOMING_NAPI_ID socket option that is supported
10in linux kernels 4.12 or higher. This socket option returns a system level
11unique ID called NAPI ID that is associated with a RX queue on which the
12last packet associated with that socket is received.
13
14This allows memcached to split the incoming flows among threads based on
15the RX queue on which they are received. Each worker thread is associated
16with a NIC HW receive queue and services all the connection requests
17received on a specific RX queue. This mapping between a memcached thread
18and a HW NIC queue streamlines the flow of data from the NIC to the
19application. In addition, an optimal path with reduced context switches is
20possible, if epoll based busy polling
21(sysctl -w net.core.busy_poll = <non-zero value>) is also enabled.
22
23This feature is enabled via a new command line parameter -N <num> or
24"--napi_ids=<num>", where <num> is the number of available/assigned NIC
25hardware RX queues through which requests associated with a connection are
26received. The number of napi_ids specified cannot be greater than the number
27of worker threads specified using -t/--threads option. If the option is
28not specified, or the conditions not met, the code defaults to round robin
29thread selection.
30
31During a normal run, each worker thread gets associated with a napi_id and
32this will establish a 1:1 mapping between the thread and queues. If a new
33napi_id is received after each thread is associated with its own napi_id
34(this can happen if num_napi_ids argument doesn't exactly match with the
35number of RX queues OR if the NIC driver goes through a reload), a stats
36error counter called 'unexpected_napi_ids' is incremented and all the
37napi_id's associated with the threads are reset.
38
39If -N option is used, but the connection requests are received from a
40virtual interface like loopback, napi_id returned can be 0. This condition
41is tracked via a stats counter called 'round_robin_fallback'.
42