A multithreaded cache for Linux, with its own encrypted protocols and a Redis-compatible door. One node to start; a cluster when you need it.

perfcached is an in-memory key-value cache. It runs as one daemon that serves requests on every core, keeps data in memory with optional write-ahead logging and snapshots, and is watched through its dashboard and Prometheus metrics.

Clients speak perfcached's own protocol: binary frames, encrypted with Noise. libperfd, the C client library (MIT), and perfcli, the command-line client, use it; for scripts, perfcli -j takes a request typed as JSON and sends it as the binary request. For applications already written against Redis, it also answers the Redis protocol (RESP2), so unmodified Redis clients such as redis-cli and hiredis work too.

$ perfcli -a "$SECRET" set cache greeting hello 60
{"stored":true}
$ perfcli -a "$SECRET" get cache greeting
{"found":true,"value":"hello","ttl":60}
$ redis-cli GET greeting
"hello"

When one machine is no longer enough, run the same binary on several: they find each other, elect a master and share the keyspace.

Why perfcached

• Its own protocol, encrypted by default. Binary frames over Noise (libsodium), with typed verbs, binary-safe values and pipelining in libperfd.
• Uses every core. Requests are served by a pool of worker threads, not one event loop. On the same 16-vCPU host, over a real network, four workers answer 2.2x Redis 8's SET rate and 1.4x its GET rate at pipeline depth 64 (measured).
• Durable when you want it. A write-ahead log plus snapshots, with a storage probe that recommends an fsync policy for your disks. A node that loses log records heals itself from memory instead of going down.
• Redis clients work too. The Redis door is guarded by an address allow-list and AUTH, and answers strings with TTLs, counters, SET ... NX locks, RedisJSON documents, pub/sub, keyspace notifications, SCAN, INFO and SLOWLOG. The full list with every difference is in REDIS-COMMANDS.md.
• Easy to watch. A dashboard (perfcached-dashboard, its own process over the native door), /metrics for Prometheus (a Grafana dashboard is in contrib/grafana), and /stats as JSON. A node raises its own alerts - a failed or healing node, lost WAL records, a full arena, an unreachable upstream, a member gone - and, in a build with mail, sends them over SMTP; in a fleet, [mail] on one node mails for all (GUIDE.md, "Mail").
• Changes while it runs. Collections are created, resized, renamed and dropped live. Memory grows under load and is given back when idle.
• A standalone edition leaves every line of cluster code out of the build, for a cache that will only ever be one node (choosing a build).

When one node is not enough

• Clusters without a coordinator. Nodes find each other - over multicast on one network segment, or through a few seed names where there is no multicast (clouds, Kubernetes) - elect a master that owns a versioned placement map, and rejoin on their own after a crash. There are no node ids or peer lists to configure.
• Five ways to place data, from every node holding everything to exactly one owner per key. Choose for availability, memory or write throughput (the modes).
• Clients that follow the fleet. libperfd keeps spare connections, fails over in one send() and can send each key straight to its owner. Redis clients that understand Redis Cluster read CLUSTER SLOTS and route themselves.
• Pub/sub across the fleet: a publish on any node reaches the subscribers on every node.
• On a LAN or in a cloud. By default membership, heartbeats and the placement map travel by multicast on one L2 segment (TTL 1; [cluster] multicast_ttl lets multicast routing carry them across segments, verified with one router in a lab rig). Where there is no multicast - AWS, GCP, Azure, Kubernetes - discovery = unicast finds the fleet through seed names, sends anything larger than the path MTU over TCP so nothing fragments, and tells a peer whose UDP is blocked from a dead one (GUIDE.md, "In a cloud or Kubernetes").

Status

The last tagged release is 0.7.3 (2026-10-25). It is young software and we want to hear how it behaves on your workload. Please open an issue for anything odd, and include the output of perfcached -V, your config (secrets removed) and the log around the event.

Known limits in 0.7.0:

• A read-through collection's password moved from [secrets] upstream into the collection, as upstream_password (each collection may now front its own Redis). 0.6.1 and later refuse the old key and perfcached -C says where it goes - move it before the restart.
• From 0.4.4 or earlier, the upgrade replaces a fleet whole: since 0.4.5 the native door speaks one binary protocol (the JSON-RPC dialect is gone) and the cluster wire epoch changed, so stop every node, start them all on the new version, and move every client to libperfd 0.3.0 or later (or the RESP door) at the same time. State from 0.4.1 or later is kept. No upgrade path from 0.3.x. From 0.4.5 on, every release is a rolling restart within the support window ("Compatibility" below); use SET IFEQ/IFNE and DELEX once every node runs 0.5.0 or later. A node keeps the identity an earlier release minted at random, and its log says how to switch to one derived from the machine.
• No lists, sets, sorted sets, streams, bitmaps, HyperLogLog or geo yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• No transactions (MULTI/EXEC/WATCH) and no general scripting: EVAL/EVALSHA run only the approved Symfony Lock scripts. An application built on either will not move over unchanged; what the Redis door answers and refuses is in REDIS-COMMANDS.md.
• Linux only (x86_64, arm64, arm32, i386), on glibc or musl.
• The write-ahead log has a ceiling, and the storage sets it: on a test host it keeps up with about 1,000,000 small SETs a second under fsync = everysec, but with 1 KB records the log's writer tops out near 180,000 a second, and in front of a disk that absorbed about 35 MB/s it fell behind at about 35,000. A write rate faster than the disk takes, or a log that fills before a snapshot completes, drops acknowledged writes from the log. They are still in memory; the node logs an ERROR, refuses writes, takes a snapshot that makes them durable again and returns to service by itself. Size the disk and the log for the write rate, or run without a log where a cache may lose writes on a crash.
• In a cluster: one L2 segment by default; across routed links only with multicast_ttl and multicast routing, not measured over WAN links; discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment, and suits a fleet of a few dozen nodes; spread mode has not yet carried production traffic; and in eager mode a delete made on one side of a network partition can be undone when the partition heals.
• Read-through from a Redis upstream carries strings, hashes and RedisJSON documents (a list, set, sorted set or stream answers WRONGTYPE); KEYS and SCAN see only what perfcached holds; proxy, shard and spread collections refuse an upstream; and tracking the upstream's changes needs every connection on one Redis server and, from 0.5.6, an upstream user allowed INFO. Write-through (0.6.2) does not bridge pub/sub, costs each write a round trip to the master, and two clients writing one key through two nodes at the same moment can leave Redis and the fleet with different winners.
• The RESP and HTTP doors are plaintext, and no TLS is planned for them for now. Under upstream_writes = through the RESP door can write to Redis: keep it on a trusted segment, behind resp_allow and [secrets] resp.

Install

Debian and Ubuntu: .deb packages for Debian 13 and 12 and Ubuntu 24.04 and 20.04 (amd64) are on the releases page, in both editions (see Choosing a build): perfcached is the clustered daemon and perfcached-standalone the single-node one. They install the same files and replace each other.

sudo apt install ./perfcached_0.6.1-1+deb13_amd64.deb
sudo apt install ./perfcached-standalone_0.6.1-1+deb13_amd64.deb   # or this one

libperfd, the client library, has packages of its own from 0.5.8: libperfd0 (the shared library, libperfd.so.0) and libperfd-dev (header, static archive, pkg-config file) as .deb, and libperfd / libperfd-dev as signed .apk for Alpine. A program built against libperfd-dev links the shared library by default; a libperfd update with the same SONAME is then an install and a restart, with no rebuild. See lib/README.md.

From source: needs a C compiler, make and libsodium.

make                 # perfcached, perfcli, perfdump, perfload, libperfd.a
                     # (+ perfcached-dashboard where libmicrohttpd-dev is installed)
make libperfd.so     # the shared library beside the archive
sudo make install    # binaries, libperfd, example config, systemd unit

Choosing a build

make menuconfig chooses which daemon make builds and saves the choice in config.mk:

edition	build it for	what you get
clustered (the default)	anything that may grow beyond one node	every cluster mode: eager, store, shard, proxy, spread
standalone	a cache that will only ever be one node	no cluster code at all: the binary is a third smaller, its static memory 7 MB less, and a [cluster] section in the config is refused rather than ignored

Without a config.mk the build is clustered. The .deb packages come in both editions: perfcached and perfcached-standalone. To choose without the menu, for scripts and CI:

make menuconfig                      # interactive (whiptail, dialog or plain prompts)
sh tools/menuconfig.sh standalone    # the same, non-interactive
make PC_EDITION=standalone           # one build, without writing config.mk

A standalone daemon reports standalone in perfcached -V and "edition" in /stats. Its tests are make check-standalone; the clustered suites need the cluster code and make check says so.

The allocator is the second thing make menuconfig asks (or make PC_ALLOC=libc|mimalloc|jemalloc): the C library's own malloc by default, or mimalloc 2.x or jemalloc linked ahead of it. The C library itself is the toolchain's - glibc on Debian, musl on Alpine. perfcached -V names a non-default allocator, and /stats carries process.allocator. After changing it on the command line rather than through make menuconfig, make clean first.

The build flavor is the third question (or make PC_BUILD=release|debug|asan): release (-O2 -g, what the packages ship) by default; debug (-Og -g3, frame pointers, the arena's own consistency checks); asan (debug plus AddressSanitizer and UBSan, their runtimes linked statically so the target needs nothing installed; glibc only). A non-release build says so in perfcached -V (perfcached 0.6.1 (abc1234, asan)), in /stats process.build and in the dashboard's header. make clean after changing it. For a test node, contrib/systemd/ has two drop-ins for perfcached.service.d/: cores.conf keeps the newest two cores, and asan.conf sends a sanitizer report there.

Alert mail is the fourth: none (the default) or libcurl (make PC_MAIL=libcurl; Debian/Ubuntu libcurl4-openssl-dev, Alpine curl-dev), which lets a node send its alerts over SMTP (STARTTLS or TLS). A build without it refuses a [mail] section rather than ignore it; its alerts are logged and kept on the dashboard all the same.

Quick start

A single node

The native door on the network, and the Redis door on localhost. Save this as /etc/perfcached/perfcached.conf (mode 640):

[daemon]
workers = 4                      # the number of cores

[memory]
arena_mb = 512                   # memory for keys and values

[secrets]
client = pick-a-client-secret    # for the native, encrypted protocol
enable = pick-an-enable-secret   # privileged commands only (perfcli -E)

[listen]
tcp = 0.0.0.0:6479               # native protocol (encrypted)
resp = 127.0.0.1:6379            # Redis protocol (optional)
resp_collections = 0:cache       # Redis database 0 is the collection "cache"

[collection cache]
buckets_log2 = 16

Three secrets, all different, none shared with anyone who does not need it: client is what applications present; enable raises one connection's privilege (perfcli -E) for the commands that change the node or the fleet - creating, resizing and dropping collections (with [daemon] allow_create = yes) and fleetstop; the cluster secret, below, only the daemons hold. Without an enable secret those commands cannot be run at all.

Check it and start it:

perfcached -f /etc/perfcached/perfcached.conf -C    # validate, show what it would run
sudo systemctl enable --now perfcached
perfcli -a pick-a-client-secret set cache greeting hello 60
perfcli -a pick-a-client-secret get cache greeting
redis-cli -p 6379 GET greeting

To serve Redis clients across a network, give the door a LAN address plus resp_allow and a [secrets] resp password; the daemon refuses to open it off-box without an allow-list.

Growing into a cluster

When one node is no longer enough, add machines. On one L2 segment the nodes find each other by multicast: it is sent with a TTL of 1 and does not cross a router unless [cluster] multicast_ttl raises it for multicast routing (PRODUCTION.md has the checklist). Add the same [cluster] section to the config on every machine and start them in any order:

[cluster]
multicast = 239.68.68.1:6480     # same group on every node
mode = eager                     # every node holds every key
collections = cache

[secrets]
client = pick-a-client-secret
enable = pick-an-enable-secret
cluster = a-different-secret     # only the daemons hold this one

Between the nodes, open port 6480 for both UDP (membership, replication, reads from peers) and TCP (large records and a new node's initial copy). UDP 6481 is optional: it keeps pub/sub relays off the membership socket, and when it is closed they travel on 6480 instead.

No multicast (AWS, GCP, Azure, Kubernetes, or a LAN that filters it): name a few members as seeds instead of a group. Every node re-resolves the names, so a seed may be down or move:

[cluster]
discovery = unicast
port = 6480
seeds = cache-0.cache, cache-1.cache, cache-2.cache
expect = 3                       # a cold start waits for all three
mode = eager
collections = cache

The ports are the same, plus TCP 6482: frames larger than the network's MTU go over a TCP connection between the two nodes instead of as IP fragments. GUIDE.md ("In a cloud or Kubernetes") covers Kubernetes and node identity. Within seconds perfcli -a <client secret> stats shows the members and which node is master. Stop a node and the others carry on; start it again and it rejoins.

Complete, validated configurations for single nodes and every cluster mode are in contrib/CONFIGURATIONS.md. Every option is annotated in contrib/perfcached.conf.example.

Cluster modes

One cluster runs one mode. The modes differ in how many nodes hold a key, and which.

mode	copies of a key	memory per node	when a node dies
eager	every node	the whole keyspace	nothing is lost
spread	K nodes (you pick K)	K/P of the keyspace	survives K-1 failures
shard	one owner, by hash	1/P	its share is unavailable
proxy	one holder, by free memory	1/P	its share is unavailable
store	nodes that read it	what each node reads	keys read only there are lost

In short: use eager for three nodes that must not lose anything, spread for four or more where you want a fixed redundancy, and shard for throughput with a client that routes by key. Durability is separate from copies. Only the node that took a write logs it, so turn on the WAL and snapshots if the whole fleet may stop at once. GUIDE.md has the trade-offs in full.

Clients

• libperfd, the C client (MIT, so it can go in proprietary code). It is encrypted and pipelined, and it follows the fleet: it keeps spare connections, fails over in one send(), and can route each key straight to its owner. See lib/README.md.
• perfcli, the redis-cli of the native protocol (PERFCLI-COMMANDS.md); for scripts, -j takes a request typed as JSON and sends it as the binary request (plain text on localhost with plaintext = loopback, encrypted otherwise). Plus perfdump and perfload for parallel backup and restore between fleets of any size or mode, and perfcached-dashboard, the status page served by its own process over libperfd (0.6.6; since 0.7.0 the daemon serves no page of its own).
• Any Redis client through the Redis door (RESP2). Cluster-aware clients read CLUSTER SLOTS and send each key to its owner.
• Asterisk: dialplan functions in contrib/asterisk.

Performance

For the best results, turn NIC multiqueue on - one queue per CPU, on the nodes and on the clients - and keep the default io_uring backend: see PRODUCTION.md, section 7. With a single receive queue, one CPU handles everything a host receives and caps it, whatever the daemon does.

redis-benchmark, unmodified, against one perfcached 0.4.0 node (4 workers) and against redis-server 8.10.1, each on the same 16-vCPU host, with the client on a second host across a real network. 20k keys of 200 bytes, 50 clients, median of 3:

	SET/s	GET/s	SET p99	GET p99
redis-server, no pipelining	41,824	46,468	1.54 ms	1.47 ms
perfcached, no pipelining	37,425	44,703	1.08 ms	1.00 ms
redis-server, pipeline 64	760,746	948,317	5.39 ms	4.98 ms
perfcached, pipeline 64	1,643,385	1,314,060	1.73 ms	1.74 ms

With no pipelining, both servers wait on the network round trip. With pipelining the servers themselves are the limit, and perfcached's workers pull ahead, at a third of Redis's tail. The lead comes from the workers, not from a cheaper command: per core, a SET and its write-ahead log record cost perfcached more than a SET costs a Redis that persists nothing, so a node with two workers can trail one Redis thread on SET. Three-node fleets through routing clients reach 3.7 to 6.7 million GET/s. A KEYS that walks 74,000 keys 50 times a second costs Redis's other clients a quarter of their GET rate; perfcached's keep theirs - the KEYS caller waits longer instead. Every table, its method and how to reproduce it: GUIDE.md and bench/RESULTS.md.

Documentation

• GUIDE.md: running and operating it. Nodes and fleets, the modes, durability, sizing, the status page, runtime collections, the tools, the clients, and all measurements.
• REDIS-COMMANDS.md: every Redis command it answers, and what an unsupported one gets.
• contrib/CONFIGURATIONS.md: ten complete configurations.
• PRODUCTION.md: the pre-deployment checklist.
• PERFCLI-COMMANDS.md: the perfcli reference.
• lib/README.md: libperfd.
• doc/perfdump-format.md: the dump format.
• ChangeLog: what each release changed.

Building and testing

make check             # every test suite (clustered edition)
make check-asan        # the same under AddressSanitizer + UBSan
make check-standalone  # the suite for the standalone edition
tools/matrix.sh        # build and test all four architectures (podman + qemu)
tools/alpine-check.sh  # Alpine (musl, gcc 15): both editions + check-standalone

CI runs lint and the fast suite on every push, and the full suite, the sanitizer suite, the four-architecture matrix and the Alpine check on every release tag. Debian and Alpine - glibc and musl - are both built and tested; bench/Containerfile.alpine is the small image.

Compatibility

perfcached is pre-1.0 and its formats may still change - but every change rolls, within one support window:

The support window. A fleet on any patch of a minor upgrades by a rolling restart to any later patch of that minor and to the next minor's releases; from an older minor, step through the last patch of each minor in between, each step a rolling restart. Mixed versions are for the roll, not for running: the daemons report their build on the wire, the status page says "mixed versions" and blinks the cards still on the older build, and a release note names any exception. The order is members first, the master last, one node at a time, reading the fleet between nodes (every node ready, its peers up, the record counts not fallen, no reconcile waiting on the node you are about to restart - the page's header says "do not restart" while one is). The client library of the same or the previous minor talks to both sides of a roll (the test and staging fleets run libperfd 0.6.0 against 0.6.x daemons).

Packages upgrade through the package manager, which leaves the running daemon in place until you restart it (PRODUCTION.md, section 8); a tarball install under /opt/perfcached-<version> upgrades through the staged build and the helper (GUIDE.md, "The upgrade helper"). The one break in the record is 0.4.5, which replaced the native protocol and the cluster wire: from 0.4.4 or earlier the fleet is stopped and started whole ("Known limits" above). State from 0.4.1 or later is kept across every upgrade since; there is no path from 0.3.x.

License

• The daemon and its tools: GPL-2.0-or-later (COPYING).
• libperfd, the client library: MIT (lib/LICENSE), so applications of any license can link it.

Each source file carries an SPDX line, and CI checks that the MIT side never includes GPL code. libsodium is ISC (see lib/NOTICE). Contributions need a DCO sign-off; see CONTRIBUTING.md.