RDMF: More consistent define names.

RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR.

RDMF: redisAssert -> serverAssert.

RDMF: OBJ_ macros for object related stuff.

RDMF: use client instead of redisClient, like Disque.

RDMF (Redis/Disque merge friendlyness) refactoring WIP 1.

Better read-only behavior for expired keys in slaves.

Slaves key expire is orchestrated by the master. Sometimes the master
will send the synthesized DEL to expire keys on the slave with a non
trivi

Better read-only behavior for expired keys in slaves.

Slaves key expire is orchestrated by the master. Sometimes the master
will send the synthesized DEL to expire keys on the slave with a non
trivial delay (when the key is not accessed, only the incremental expiry
algorithm will expire it in background).

During that time, a key is logically expired, but slaves still return
the key if you GET (or whatever) it. This is a bad behavior.

However we can't simply trust the slave view of the key, since we need
the master to be able to send write commands to update the slave data
set, and DELs should only happen when the key is expired in the master
in order to ensure consistency.

However 99.99% of the issues with this behavior is when a client which
is not a master sends a read only command. In this case we are safe and
can consider the key as non existing.

This commit does a few changes in order to make this sane:

1. lookupKeyRead() is modified in order to return NULL if the above
conditions are met.
2. Calls to lookupKeyRead() in commands actually writing to the data set
are repliaced with calls to lookupKeyWrite().

There are redundand checks, so for example, if in "2" something was
overlooked, we should be still safe, since anyway, when the master
writes the behavior is to don't care about what expireIfneeded()
returns.

This commit is related to #1768, #1770, #2131.

show more ...

Over 80 chars comment trimmed in pfcountCommand().

Remove warnings and improve integer sign correctness.

PFSELFTEST: less false positives.

This is just a quickfix, for the nature of the test the right way to fix
it is to average the error of N runs, since otherwise it is always
possible to get a false

PFSELFTEST: less false positives.

This is just a quickfix, for the nature of the test the right way to fix
it is to average the error of N runs, since otherwise it is always
possible to get a false positive with a bad run, or to minimize too much
this possibility we may end testing with too much "large" error ranges.

show more ...

Correct the HyperLogLog stale cache flag to prevent unnecessary computations.

Set the MSB as documented.

Speedup hllRawSum() processing 8 bytes per iteration.

The internal HLL raw encoding used by PFCOUNT when merging multiple keys
is aligned to 8 bits (1 byte per register) so we can exploit this to
im

Speedup hllRawSum() processing 8 bytes per iteration.

The internal HLL raw encoding used by PFCOUNT when merging multiple keys
is aligned to 8 bits (1 byte per register) so we can exploit this to
improve performances by processing multiple bytes per iteration.

In benchmarks the new code was several times faster with HLLs with many
registers set to zero, while no slowdown was observed with populated
HLLs.

show more ...

Speedup SUM(2^-reg[m]) in HyperLogLog computation.

When the register is set to zero, we need to add 2^-0 to E, which is 1,
but it is faster to just add 'ez' at the end, which is the number of
regist

Speedup SUM(2^-reg[m]) in HyperLogLog computation.

When the register is set to zero, we need to add 2^-0 to E, which is 1,
but it is faster to just add 'ez' at the end, which is the number of
registers set to zero, a value we need to compute anyway.

show more ...

PFCOUNT support for multi-key union.

HyperLogLog low level merge extracted from PFMERGE.

HyperLogLog invalid representation error code set to INVALIDOBJ.

PFDEBUG TODENSE added.

Converts HyperLogLogs from sparse to dense. Used for testing.

User-defined switch point between sparse-dense HLL encodings.

PFSELFTEST improved with sparse encoding checks.

PFDEBUG ENCODING added.

Set HLL_SPARSE_MAX to 3000.

After running a few benchmarks, 3000 looks like a reasonable value to
keep HLLs with a few thousand elements small while the CPU cost is
still not huge.

This covers all

Set HLL_SPARSE_MAX to 3000.

After running a few benchmarks, 3000 looks like a reasonable value to
keep HLLs with a few thousand elements small while the CPU cost is
still not huge.

This covers all the cases where the dense representation would use N
orders of magnitude more space, like in the case of many HLLs with
carinality of a few tens or hundreds.

It is not impossible that in the future this gets user configurable,
however it is easy to pick an unreasoable value just looking at savings
in the space dimension without checking what happens in the time
dimension.

show more ...

Error message for invalid HLL objects unified.

PFMERGE fixed to work with sparse encoding.

Correctly replicate PFDEBUG GETREG.

Even if it is a debugging command, make sure that when it forces a
change in encoding, the command is propagated.

Added assertion in hllSparseAdd() when promotion to dense occurs.

If we converted to dense, a register must be updated in the dense
representation.