fix dyn-maxbytes tests

tests were handling undef values magically... thankfully they still pass after
fixing the inputs.

fix test RAM bloat and more 32bit offsets.

when doing tests with long strings the results are buffered... so doing
thousands of tests with long strings was using more than a gig of ram. now we
just

fix test RAM bloat and more 32bit offsets.

when doing tests with long strings the results are buffered... so doing
thousands of tests with long strings was using more than a gig of ram. now we
just summarize if any were different.

also fixes more offset stuff.

show more ...

update item_size_max tests

more rules around item sizes now. slab chunks automatically adjust if
requesting memory > 1M

slabs reassigns works with chunks and chunked items.

also fixes the new LRU algorithm to balance by total bytes used rather than
total chunks used, since total chunks used isn't tracked for multi-ch

slabs reassigns works with chunks and chunked items.

also fixes the new LRU algorithm to balance by total bytes used rather than
total chunks used, since total chunks used isn't tracked for multi-chunk
items.

also fixes a bug where the lru limit wasn't being utilized for HOT_LRU

also some cleanup from previous commits.

show more ...

CLRF in binprot was inverted.

now extends data in the chunk rather than overwriting the end.

chunked append/prepend now work.

This is also the only actual tests of append and prepend in the codebase :/
t/binary.t had some simple tests but no others were ever written.

startup options for chunked items.

has spent some time under performance testing. For larger items there's less
than 5% extra CPU usage, however the max usable CPU when using large items is
1/10th o

startup options for chunked items.

has spent some time under performance testing. For larger items there's less
than 5% extra CPU usage, however the max usable CPU when using large items is
1/10th or less before you run out of bandwidth. Mixed small/large items will
still balance out.

comments out debugging (which must be removed for release).

restores defaults and ensures only t/chunked-items.t is affected.

dyn-maxbytes and item_size_max tests still fail.

append/prepend aren't implemented, sasl needs to be guarded.

slab mover needs to be updated.

show more ...

chunked item second checkpoint

can actually fetch items now, and fixed a few bugs with storage/freeing.

added fetching for binprot.
added some basic tests.

many tests still fail for various reason

chunked item second checkpoint

can actually fetch items now, and fixed a few bugs with storage/freeing.

added fetching for binprot.
added some basic tests.

many tests still fail for various reasons, and append/prepend isn't fixed yet.

show more ...

make test not flaky on 32bit OS.

since slab classes don't align necessarily. should maybe use the explicit slab
sizing feature to make 32/64 bit tests more stable.

too close to the line for forcing evictions.

fix flakiness of slabs-reassign2.t test.

slabs_automove=2 is fairly dumb and should be replaced with something better.
I'm guessing a kickoff from the LRU juggler if free chunks is zero, rather
than

fix flakiness of slabs-reassign2.t test.

slabs_automove=2 is fairly dumb and should be replaced with something better.
I'm guessing a kickoff from the LRU juggler if free chunks is zero, rather
than inline in the lru_pull_tail call.

This test was checking rescues/inline evictions and other newer features of
the slab rebalancer. So now that's the central point rather than racing the
slab mover thread. Tests now seem reliable.

show more ...

make watcher.t be even less flaky.

use X macros to remove several struct iterations

Adding new stats would require updating in too many locations. No longer have
to. Could extend this a bit to do the actual stats printing, but for cl

use X macros to remove several struct iterations

Adding new stats would require updating in too many locations. No longer have
to. Could extend this a bit to do the actual stats printing, but for clarity's
sake I'll decide on that later.

show more ...

add get_flushed counter, fix expired_unfetched

get_flushed ticks when directly fetching something that classified as expired.
LRU crawler was ticking expired_unfetched for unfetched _flushed_ values

add get_flushed counter, fix expired_unfetched

get_flushed ticks when directly fetching something that classified as expired.
LRU crawler was ticking expired_unfetched for unfetched _flushed_ values.
should no longer do that.

also removes get_expired from the global stats list. I missed that on review.

show more ...

bump binary.t

cache_memlimit command for tuning runtime maxbytes

Allows dynamically increasing the memory limit of a running system, if memory
isn't being preallocated.

If `-o modern` is in use, can also dynamic

cache_memlimit command for tuning runtime maxbytes

Allows dynamically increasing the memory limit of a running system, if memory
isn't being preallocated.

If `-o modern` is in use, can also dynamically lower memory usage. pages are
free()'ed back to the OS via the slab rebalancer as memory is freed up. Does
not guarantee the OS will actually give the memory back for other applications
to use, that depends on how the OS handles memory.

show more ...

add -o watcher_logbuf_size=N worker_logbuf_size=N

the defaults should just about always be used, but this is now tunable for
completeness. Also handy for writing tests.

This also hopefully makes th

add -o watcher_logbuf_size=N worker_logbuf_size=N

the defaults should just about always be used, but this is now tunable for
completeness. Also handy for writing tests.

This also hopefully makes the watcher.t test not flaky anymore.

show more ...

treat and print item flags as unsigned int

most of the code would parse and handle flags as unsigned int, but passed into
alloc functions as a signed int... which would then continue to print it as

treat and print item flags as unsigned int

most of the code would parse and handle flags as unsigned int, but passed into
alloc functions as a signed int... which would then continue to print it as
unsigned up until a change made in 2007. Now treat it fully as unsigned and
print as unsigned.

show more ...

make watcher test less flaky.

as usual, need a better way of doing this :/

memcached idle connection killer

misc cleanups for logger code.

tests pass now.

logger endpoints for first release

swapping "RAWCMDS" for internal hooks on when items are fetched or stored.

This doesn't map 1:1 with commands, ie: a store is internally a fetch then
store, so tw

logger endpoints for first release

swapping "RAWCMDS" for internal hooks on when items are fetched or stored.

This doesn't map 1:1 with commands, ie: a store is internally a fetch then
store, so two log lines are generated. An ascii multiget one make one log line
per key fetched.

It's a good place to start. Need to come back and refactor parts of logger.c
again, then convert all prints in the codebase to log entries.

show more ...

convert eviction log to new process

add LOGGER_LOG() macro wrapper for callers.

eviction logging is now using the "more final" method, where data is copied
into structs embedded into the per-worker

convert eviction log to new process

add LOGGER_LOG() macro wrapper for callers.

eviction logging is now using the "more final" method, where data is copied
into structs embedded into the per-worker circular buffer. This moves the
expensive snprintf calls to the background thread, and also allows making the
logging format switchable.

also switched the format to key=value. it's still largely readable but much
easier to parse.

show more ...

logger: add a uri encoder for printing keys

makes a map on start of bgthread to speed up conversion a little.

some basic tests for the watcher.