A caching library built for moving fast without breaking things. GCache lets you rapidly add new caching use cases while maintaining structure and runtime control guardrails—so you can ramp up gradually, kill a bad cache instantly, and have full observability into what's cached across your system.
Most caching libraries give you a key-value store and leave the rest to you. GCache takes a different approach:
- Opinionated structure — Enforced key format (
key_type+ ID + use case, e.g.,user_id:123) keeps your caching organized and enables the features below - Runtime controls — Enable/disable caching per request, ramp from 0-100% per use case, adjust configuration without redeploying
- Targeted invalidation — Invalidate all cache entries for a
key_type+ ID (e.g., all caches for a specific user, org, or project) with one call - Full observability — Prometheus metrics out of the box, broken down by use case and
key_type
pip install gcacheRequires Python 3.10+
from gcache import GCache, GCacheConfig, GCacheKeyConfig, CacheLayer
# Create the cache instance (singleton)
gcache = GCache(GCacheConfig())
# Decorate your function
@gcache.cached(
key_type="user_id",
id_arg="user_id",
use_case="GetUser",
default_config=GCacheKeyConfig(
ttl_sec={CacheLayer.LOCAL: 60, CacheLayer.REMOTE: 300},
ramp={CacheLayer.LOCAL: 100, CacheLayer.REMOTE: 100},
),
)
async def get_user(user_id: str) -> dict:
return await db.fetch_user(user_id) # Your expensive operation
# Use it — caching only happens inside enable() blocks
with gcache.enable():
user = await get_user("123") # Cache key: urn:gcache:user_id:123#GetUserThat's it. The function works normally outside enable() blocks, and caches results inside them.
GCache uses a multi-layer read-through cache:
Request
│
▼
┌─────────────────┐
│ LOCAL CACHE │ ◄─── Hit? Return immediately
│ (in-memory) │
└────────┬────────┘
│ Miss
▼
┌─────────────────┐
│ REDIS CACHE │ ◄─── Hit? Store in local, return
│ (distributed) │
└────────┬────────┘
│ Miss
▼
┌─────────────────┐
│ YOUR FUNCTION │ ◄─── Execute, store in both caches, return
└─────────────────┘
Local cache is fast but per-instance. Redis is shared across your fleet. Use both for best performance, or just local if you don't need Redis.
GCache constructs structured cache keys in URN format:
urn:prefix:key_type:id?arg1=val1&arg2=val2#use_case
For example: urn:gcache:user_id:123?page=1#GetUserPosts
This structure is useful for:
- Debugging — Keys are human-readable when inspecting Redis
- Grouping — All caches for a
key_type:idpair share a common prefix, making it easy to find related entries - Targeted invalidation — The structure enables invalidating all entries for a specific
key_type+ ID
Caching doesn't happen automatically—you control when it's active:
-
enable()context — Caching only happens insidewith gcache.enable():blocks. Outside of them, your function runs normally. This lets you disable caching during write operations to avoid stale reads. -
ramppercentage — Each cache layer has a ramp from 0-100%. At 50%, half the requests use the cache, half go straight to the source. Start at 0% when adding a new use case, then ramp up as you gain confidence. -
Dynamic config — The config provider runs on each request, so you can adjust TTLs or ramp percentages without redeploying.
GCache requires you to explicitly enable caching with with gcache.enable():. This is intentional.
Caching in write paths can cause subtle bugs—a stale read might get cached right before a write, leading to inconsistent data. By requiring explicit opt-in, GCache forces you to consciously decide where caching is safe:
# Read path — caching is safe
with gcache.enable():
user = await get_user(user_id)
# Write path — no caching, function runs normally
await update_user(user_id, new_data)
await gcache.ainvalidate("user_id", user_id)This design prevents accidental caching in dangerous places.
For dynamic control, provide a config provider when creating GCache. This lets you adjust caching behavior without redeploying:
from gcache import GCache, GCacheConfig, GCacheKeyConfig, GCacheKey, CacheLayer
async def config_provider(key: GCacheKey) -> GCacheKeyConfig | None:
# Fetch from your config source: LaunchDarkly, database, config file, etc.
config = await config_service.get_cache_config(key.use_case)
if config is None:
return None # Fall back to default_config on the decorator
return GCacheKeyConfig(
ttl_sec={CacheLayer.LOCAL: config.local_ttl, CacheLayer.REMOTE: config.remote_ttl},
ramp={CacheLayer.LOCAL: config.local_ramp, CacheLayer.REMOTE: config.remote_ramp},
)
gcache = GCache(GCacheConfig(cache_config_provider=config_provider))This enables:
- Kill switches — Set ramp to 0% to instantly disable a problematic cache
- Gradual rollout — Start at 10%, monitor metrics, increase to 100%
- Per-use-case tuning — Different TTLs and ramp percentages for different use cases
If a use case sets invalidation_tracking=True and a remote ttl_sec above
MAX_TRACKED_TTL_SECONDS (4 hours), writes now raise TrackedTTLExceedsWatermark (also a
ValueError). This was previously accepted, and GCacheKeyConfig still types ttl_sec as a
plain int — the bound is a property of invalidation, not of the type.
What you will see if you are affected. aput/put raise. A @cached miss does not —
CacheController catches the write error, increments gcache_error_counter, and returns the
fallback value. So the caller keeps working and the cache silently never populates: the
symptom is a use case whose hit rate sits at zero with a rising gcache_error_counter, not an
outage. Check that counter, not your latency.
Migration: either lower ttl_sec to 4 hours or less, or turn off invalidation_tracking
for that key. An untracked key may use any TTL — it has no watermark to outlive.
Why not just cap it silently: a cap gives you a shorter TTL than you configured and hides the misconfiguration from the read guards below, so it would surface nowhere.
Reads are guarded too, in two ways, because a write cap only binds entries this process writes and the keyspace is shared:
- a tracked JSON entry whose envelope declares a lifetime over 4 hours (
MAX_TRACKED_TTL_SECONDS) is a miss (gcache_miss_counter{reason="lifetime_exceeds_watermark"}) - a tracked entry of either framing that is older than 4 hours (
MAX_TRACKED_TTL_SECONDS) is a miss (reason="age_exceeds_watermark"). This is the one that covers legacy pickle entries, which carry noexpiresAtMsfor the first check to read. An entry can only reach that age if its TTL exceeded 4 hours, so a correctly configured key never triggers it.
The invariant behind all of this: invalidation marks entries stale by writing a watermark
that lives 5 hours (WATERMARK_TTL_SECONDS). An entry outliving its watermark stops looking stale the moment the
watermark expires, and an invalidated value resurrects for the rest of its own TTL —
silently. Go has enforced both halves from the start (maxEntryTTL on write,
ResultDistrusted on read); Python previously enforced neither, so the two clients disagreed
about the same key.
GCacheConfig(urn_prefix="") raises EmptyUrnPrefixNotSupported (also a ValueError, so an
existing except ValueError around construction still catches it).
If you pass it today, your keys are not what you think. It used to be silently ignored, so
the previous prefix stayed in force — "urn" by default. The error tells you that; it does not
change the keys you were actually getting.
Migration: omit urn_prefix to keep the default "urn", or pass a non-empty prefix. There
is no configuration that reproduces the old behaviour, because the old behaviour was "use the
previous value", not "use no prefix".
Why rejected rather than made to work: an empty prefix writes into an unnamespaced key space that no namespaced deployment reads, so the cache never hits and nothing errors at write time. The Go client refuses it at construction for the same reason.
That is a different problem from the encoding question below, which is an encoding defect with a fix — align the two clients and every non-empty prefix interoperates. An empty one still would not, because that divergence is structural. It is the one case that survives the fix, which is why it gets an error instead of a caveat.
The decorator handles both sync and async functions automatically.
@gcache.cached(
key_type="user_id", # What kind of entity is this?
id_arg="user_id", # Which argument contains the ID?
use_case="GetUserProfile", # Identifies this specific caching use case
)
async def get_user_profile(user_id: str) -> dict:
...Tip: Always define use_case explicitly. It identifies the specific caching scenario (e.g., GetUserProfile, ListOrgProjects) and appears in cache keys, metrics, and logs. It defaults to module.function_name, but an explicit name ensures consistency if you refactor your code.
Options for mapping function arguments to cache keys.
Specifies which argument contains the entity ID for the cache key.
String form — use when the argument itself is the ID:
id_arg="user_id" # user_id argument is the IDTuple form — use when the ID needs to be extracted from an object:
id_arg=("user", lambda u: u.id) # Extract ID from User objectConverts complex arguments to strings for the cache key. Only needed for non-primitive types.
arg_adapters={
"filters": lambda f: f.to_cache_key(), # Complex object
"page": str, # Simple conversion
}Excludes arguments that don't affect the cached result.
ignore_args=["db_session", "logger"]@gcache.cached(
key_type="user_id",
id_arg=("user", lambda u: u.id),
arg_adapters={"filters": lambda f: f.to_cache_key()},
ignore_args=["db_session", "logger"],
)
async def search_user_posts(
user: User,
filters: SearchFilters,
page: int,
db_session: Session,
logger: Logger,
) -> list[Post]:
...
# Cache key: urn:gcache:user_id:123?filters=active&page=2#SearchUserPostsThe id_arg becomes :123, arg_adapters produce ?filters=active&page=2, and ignore_args are excluded.
@gcache.cached(key_type="org_id", id_arg="org_id", use_case="GetOrgSettings")
def get_org_settings(org_id: str) -> dict: # No async needed
return db.query(...)Under the hood, sync functions run through a thread pool to avoid blocking the event loop. This adds some overhead, so prefer async functions when possible for better performance.
By default a value is stored as a Python pickle, which only Python can read. envelope=
switches to a JSON framing the Go client also understands, so both can share one entry and
one invalidation:
from gcache import Envelope, JsonSerializer
@gcache.cached(
key_type="session_id",
id_arg="session_id",
use_case="session-identity",
envelope=Envelope.JSON,
serializer=JsonSerializer(), # required: JSON carries a serialized payload
track_for_invalidation=True, # so another language's invalidation reaches this entry
)
async def get_session(session_id: str) -> dict:
return await db.fetch(session_id)Four constraints:
- Only JSON-representable values. The payload is a string on the wire, so the serializer
has to be able to produce and restore one.
JsonSerializercovers dicts, lists and scalars; anything else needs your ownSerializer. serializer=is the same trap. Adding one to a live use case is undetectable in a pickle envelope: a pod on the older code hands the raw serialized payload back to its caller instead of the value, with nothing logged. The JSON case is caught — the reader knows it needs a serializer and treats the entry as a miss — but the pickle one cannot be. Newuse_casefor that too.- Key components are interpolated raw, in both clients. Neither Python nor Go
percent-encodes, so a component containing
:,#,?,&or=can collide with a differently-structured key that renders identically —id="a#u2"withuse_case="u"renders the same asid="a"withuse_case="u2#u". Both clients agree, so this is a property of the key grammar rather than a divergence, but it means component values should not carry the grammar's own delimiters. - Never flip this on a live use case. A rolling deploy runs both pod generations at
once: an old pod (pickle, no serializer) treats a JSON entry as a miss and writes pickle
over it, and a new pod refuses that pickle and writes JSON again. Each destroys the
framing the other needs, so the key's hit rate sits near zero for the whole rollout.
Migrate under a new
use_case— the two generations then use different keys and never fight.
Reads sniff the framing they actually find rather than trusting envelope=, so a JSON key
reads a JSON entry whoever wrote it. The one exception is that a JSON key refuses to
unpickle: unpickling executes arbitrary code, and the reader cannot tell a migration
leftover from a payload injected by anyone with keyspace access.
Note that track_for_invalidation=True only reaches the Redis layer. LocalCache does not
consult watermarks, so an invalidation from another language does not clear a Python pod's
in-process copy until its local TTL expires — keep the local TTL short (or the local ramp at
0) for a use case shared across languages.
hash_component() (Go: HashComponent()) returns lowercase hex of SHA-256 over the UTF-8
bytes, identically in both clients — the shared conformance corpus pins their agreement.
from gcache import GCacheKey, hash_component
key = GCacheKey(
key_type="session_external_id",
# project and run stay readable from redis-cli; only the caller-supplied id is opaque.
id=f"{project_id}:{run_id}:{hash_component(external_id)}",
use_case="ingest::session_by_external_id",
)Use it for a component that must not sit in a Redis key in the clear — an external id that
may be an email, an api key. Keys appear in SCAN, --bigkeys, slowlog, MONITOR and any
key-sampling metrics, which is a wider audience than the store the value came from.
Hash the component, not the whole id. Since the result goes in as an ordinary component,
every path — aget, aput, adelete, ainvalidate, the watermark key — agrees with no
further work, and the unhashed parts stay greppable.
Not salted and not truncated, deliberately: a salt cannot be shared across processes without
new configuration, and truncating trades collision resistance — two ids answering to one
entry is a wrong answer, not a slow one. It raises UnhashableKeyComponent for a value with
no UTF-8 encoding (only a lone surrogate reaches that), because Go would hash different bytes
for the same input and the two clients would silently occupy different key spaces.
@cached fits whenever the value is a pure function of a call's arguments. A shared cache
often is not: the key comes from data that is nobody's parameter, and the value is something
the caller already fetched. aget/aput (and their sync get/put) take a key you build:
from gcache import Envelope, GCacheKey, JsonSerializer
def identity_key(project_id: str, session_id: str) -> GCacheKey:
return GCacheKey(
key_type="session_id",
id=f"{project_id}:{session_id}",
use_case="session-identity",
envelope=Envelope.JSON,
serializer=JsonSerializer(),
invalidation_tracking=True,
)
# Read. The fallback runs only on a miss, so the caller can reuse whatever it fetched --
# a plain get would make it read its source of truth twice.
async def resolve(project_id: str, session_id: str) -> dict:
fetched: dict | None = None
async def load() -> dict:
nonlocal fetched
fetched = await db.fetch_session(project_id, session_id)
return fetched
with gcache.enable():
return await gcache.aget(identity_key(project_id, session_id), load)
# Write. For priming an entry ANOTHER process will read, when this one already has the
# value and does not want the read that would otherwise populate it.
with gcache.enable():
await gcache.aput(identity_key(project_id, session_id), {"session_id": session_id})Both honour enable() and the use case's ramp, exactly as the decorator does — a write
outside an enable() block, or for a use case ramped to 0, does nothing.
Four things to know:
envelopeandserializerare not part of the key.key_type,id,argsanduse_caseare, so getting one of those wrong just means the two participants never see each other's entries.argsis sorted by name in the constructor, as@cachedand Go'sValueKeyboth do, so the order you pass does not matter andkey.argsreads back in a different order than you gave it. Getting the envelope wrong is worse: both then share one key with incompatible framing, each overwrites the other, and neither can read what it finds. Within one process the library now catches it — a direct key whose envelope contradicts a@cacheddeclaration on the sameuse_caseraises — but across languages it cannot.aputraises on a cache-layer failure;agetdoes not.aputmatchesadeleteandainvalidatein propagating, so a caller on a request path decides what to do with a Redis timeout rather than having "prime" silently read as best-effort.agetis the opposite and deliberately so: it catchesGCacheError, logs, incrementsgcache_error_counterand reads through uncached, because a cache must not be able to fail a request. Plan error handling around both — atrythat wraps only the read is wrapping the half that cannot raise.- A prime is sampled by the ramp, like a read.
_should_cachecallsrandom()per invocation, per layer, so a use case at ramp 50 drops about half its primes andaputstill returns normally. On a read that costs one uncached call; on a prime it throws away work already done and the entry another process waits for never appears. Ramp a shared use case to 100 or 0, not through the middle. - A prime inside an active invalidation window is lost silently — on the Redis layer.
The entry is written with
createdAtMsbelow the watermark, so remote reads find it stale until the window closes and a read rewrites it. That is the invalidation doing its job, butaputstill returns normally. Go behaves the same way. The local layer never reads watermarks, so a lateragetin the same process returns the primed value regardless — keep the local ramp at 0 for a shared use case, as above.
JsonSerializer leaves the payload schema as something each language writes by hand, and
that is where cross-language caches actually break: not in the framing, which is tested, but
in a field one side renamed, retyped, or made optional. Review of the first shared use case
here turned up six such divergences, every one found by a person rather than a test.
ProtoSerializer takes a generated protobuf message instead, so one .proto defines the
payload for every language — and pairs with Envelope.PROTO, which stores it in the binary
wire format:
from gcache import Envelope, ProtoSerializer
from libs.python.schemas.cache.proto import session_identity_pb2
@gcache.cached(
key_type="session_id",
id_arg="session_id",
use_case="session-identity",
envelope=Envelope.PROTO,
serializer=ProtoSerializer(session_identity_pb2.SessionIdentity),
track_for_invalidation=True,
)
async def get_session(session_id: str) -> session_identity_pb2.SessionIdentity: ...Requires the extra: pip install 'gcache[protobuf]'. Importing gcache without it is fine;
only constructing ProtoSerializer raises. The Go counterpart is go/protocodec.Proto.
Measured on a small message (a uuid string and a timestamp), against the same message encoded as JSON text inside the JSON envelope:
| JSON text in JSON envelope | binary in PROTO envelope | |
|---|---|---|
| serialize | 2.23 µs | 0.12 µs |
| parse | 6.48 µs | 0.29 µs |
| stored entry | 204 B | 69 B |
Most of the size win is the envelope, not the payload: the JSON envelope costs ~102 bytes against the binary one's 18.
Binary also removes a class of bug rather than testing for it. A JSON encoding of a
protobuf message spells every field twice — session_id and sessionId — and both readers
accept either, so a writer emitting the wrong one produces two wire forms for one key and
nothing fails. Binary carries field numbers, so there is no spelling to disagree about,
and unknown fields are skipped by
protobuf itself rather than by an option each language has to remember to set.
The message type is not on the wire, so the two clients must agree on it out of band.
Field numbers carry no names, so a payload written as MessageA and read as MessageB does
not fail — protobuf skips the unknown fields and returns a zero-valued message, which reads
as a hit. Python's wire_identity() catches the mismatch inside one process, but nothing
catches it across languages or deployments. The use_case is the contract: both clients
declaring it must name the same message, and the shared conformance corpus is where that
pairing is pinned. Carrying the name in the envelope was considered and rejected: a full
message name would multiply the 18-byte overhead that is the reason to choose PROTO, and it
would break on a legitimate message rename.
What you give up is inspectability. A JSON-enveloped entry can be read with redis-cli GET, piped through jq, or parsed inside Redis by a Lua script using cjson. A PROTO entry
is opaque to all three — payload and metadata. No general inspector exists yet:
go/cmd/gcachectl round-trips only the fixed descriptorpb.FileOptions the cross-language
suite uses, and prints no envelope metadata, so pointed at any other message it prints an
empty or wrong value. If being able to eyeball an entry matters more than its size, use
Envelope.JSON, whose serializer must return text — see below.
Envelope.JSON carries text, and Envelope.PROTO carries bytes. A serializer used with
the JSON envelope must return str; a bytes payload is refused at the write with an error
naming Envelope.PROTO. So a JSON entry always decodes to str and a PROTO entry to
bytes — the payload's type follows the declared envelope.
gcache = GCache(GCacheConfig())from gcache import RedisConfig
gcache = GCache(
GCacheConfig(
redis_config=RedisConfig(
host="redis.example.com",
port=6379,
password="secret",
),
)
)For dynamic credentials, token refresh, or connection pooling:
import threading
from redis.asyncio import Redis
def make_redis_factory():
local = threading.local()
def factory() -> Redis:
if not hasattr(local, "client"):
token = fetch_token_from_vault()
local.client = Redis.from_url(f"redis://:{token}@redis:6379")
return local.client
return factory
gcache = GCache(
GCacheConfig(
redis_client_factory=make_redis_factory(),
)
)Important: Custom factories must use thread-local storage. Each thread needs its own client.
Reachable two ways — RedisConfig.redis_py_options and a custom factory — and it is a
process-wide decision, because one GCache uses one client for every use case.
The option makes redis-py decode every reply as UTF-8. A pickle blob starts 0x80, which is
not a valid UTF-8 start byte, so client.get raises UnicodeDecodeError inside redis-py,
before gcache sees the value. That error escapes the read path: gcache_error_counter moves,
the fallback re-runs, and the entry is not rewritten — so every read of every
Envelope.PICKLE use case in that process fails for its full TTL. It does not self-heal,
because a rewrite would fail identically on the next read.
"All my use cases are Envelope.JSON" is not sufficient, and an earlier version of this
section wrongly said it was. Reads sniff the framing rather than trusting the declaration
— that is exactly what makes a no-flag-day migration possible — so a JSON-declared key is
expected to meet legacy pickle values for as long as any remain in the keyspace. Those reads
fail at the client, before gcache sees a byte, and never heal.
So the real condition is stronger: turn it on only when no pickle value can be reached on
that client at all — every use case on it is Envelope.JSON and no pre-migration pickle
entries survive for those keys. In practice that means after a full TTL has elapsed since the
last pickle writer stopped.
gcache logs a warning once per RedisCache, on the first read of any kind through a text-mode
client — deliberately not gated on the key's declared envelope, since the declaration is the
thing that cannot be trusted here. It cannot fix the configuration for you.
This is new as of the envelope work. Before decode() accepted a str, the option broke
JSON reads too, so nobody could turn it on — making the JSON path work is what made the
pickle path reachable.
When data changes, you need to invalidate the cache. GCache makes this easy with targeted invalidation.
# Mark the function for invalidation tracking
@gcache.cached(
key_type="user_id",
id_arg="user_id",
track_for_invalidation=True, # Enable this
)
async def get_user(user_id: str) -> dict:
...
# When data changes, invalidate all cached entries for that key_type + ID
await gcache.ainvalidate(key_type="user_id", id="12345")
# Sync version
gcache.invalidate(key_type="user_id", id="12345")This invalidates all cache entries for that key_type + ID—every use case, every argument combination.
If a read happens right before a write, the stale data might get cached. Use a future buffer:
await gcache.ainvalidate(
key_type="user_id",
id="12345",
future_buffer_ms=5000, # Also invalidate anything cached in the next 5 seconds
)The buffer is capped at one hour (MAX_FUTURE_BUFFER_SECONDS); a larger value raises
ValueError, as does a negative one or anything that is not an int of milliseconds.
The ceiling is not arbitrary — it pairs with the 4-hour tracked-TTL cap so that
buffer + ttl always fits inside the 5-hour watermark lifetime. A watermark has to outlive
every entry it suppresses; if it expires first, an entry written inside the buffer window
becomes readable again. Splitting the sum into two caps means each is enforced locally
against a value its own caller owns — the write path checks the TTL, invalidate checks the
buffer — so neither needs to know the other's number at call time.
Seconds is the intended scale here: the buffer covers replication lag, not a maintenance window.
Ramping a use case to 0%, or leaving the enable() context, stops reads and writes but does
not clear watermarks — an invalidation written just before the switch survives in Redis
for the watermark key's own 5-hour TTL. Only flushall() or that TTL expiring removes it.
It is usually harmless: the watermark suppresses entries created at or before it, so each
affected entity costs one miss and then repopulates. future_buffer_ms is the case to know
about — a watermark set in the future also blocks write-back (_exec_fallback re-puts only
once the watermark is in the past), so those entities keep missing until the buffer elapses.
For testing or emergencies:
gcache.flushall() # Sync
await gcache.aflushall() # AsyncGCache exports Prometheus metrics automatically:
Labels are listed in full per metric, because they are not uniform -- a query written against the wrong label set returns nothing rather than erroring.
| Metric | Type | Labels |
|---|---|---|
gcache_request_counter |
Counter | use_case, key_type, layer |
gcache_miss_counter |
Counter | use_case, key_type, layer, reason |
gcache_disabled_counter |
Counter | use_case, key_type, layer, reason |
gcache_error_counter |
Counter | use_case, key_type, layer, error, in_fallback |
gcache_invalidation_counter |
Counter | key_type, layer |
gcache_get_timer |
Histogram | use_case, key_type, layer |
gcache_fallback_timer |
Histogram | use_case, key_type, layer |
gcache_serialization_timer |
Histogram | use_case, key_type, layer, operation |
gcache_size_histogram |
Histogram | use_case, key_type, layer |
gcache_invalidation_counter is the exception worth knowing -- it carries no use_case:
invalidation is keyed on
(key_type, id) and carries no use case, so it cannot be broken down or joined by
use_case. An earlier version of this section claimed "all metrics include use_case and
key_type", which was false for exactly that metric.
reason values.
gcache_miss_counter: empty for an ordinary miss, otherwise undecodable,
json_without_serializer, lifetime_exceeds_watermark, envelope_expired,
age_exceeds_watermark, reversed_envelope_timestamps, created_at_beyond_future_buffer,
divergent_payload_encoding, unloadable_payload, unreadable_watermark,
non_finite_watermark.
unreadable_watermark self-heals in this client: a watermark that is not a number at
all is deleted, so the next Python read caches again. The Go client's Client interface has
no delete, so a Go-only deployment keeps missing for the watermark key's own 5-hour TTL — in
a mixed deployment one Python read repairs the key for both. non_finite_watermark heals in
neither: nan/inf parse as numbers, so they could be a format this client does not
understand yet, and deleting one could resurrect what a newer writer had invalidated.
gcache_disabled_counter: ramped_down, context, server_down, missing_config,
config_error.
You can add a prefix to avoid collisions:
GCacheConfig(
metrics_prefix="myapp_", # Metrics become myapp_gcache_request_counter, etc.
)GCache is designed to fail open. If Redis is down or an error occurs:
- The underlying function executes normally
- The error is logged and counted in
gcache_error_counter - Your request succeeds (just without caching)
This means a cache failure never breaks your application.
A corrupt pickle and a serializer load failure used to raise out of
RedisCache.get, which incremented gcache_error_counter and re-ran the fallback without
writing back — so the bad entry stayed for its whole TTL and failed every read. Both are now
a miss that rewrites the entry, counted as gcache_miss_counter{reason="..."}.
An operator alerting on gcache_error_counter loses those two signals; the equivalent
alert is a non-empty reason on the miss counter. Existing queries keep matching — Prometheus
leaves unnamed labels unconstrained, so gcache_miss_counter{use_case="x"} still selects the
new series. What changes is what they mean: such a query now also counts degraded misses.
Add reason="" where you want the old population, not to make the selector work.
reason |
What was found |
|---|---|
undecodable |
The envelope itself could not be parsed (corrupt, or an unknown version) |
json_without_serializer |
A JSON or PROTO entry on a key that declares no Serializer — typically a rolling deploy where new pods have started writing an encoded framing. (_decode_proto sets is_json=True, so a PROTO entry reaches this guard too; the label predates the framing.) |
lifetime_exceeds_watermark |
A tracked JSON or PROTO entry whose envelope declares a lifetime longer than the 4-hour tracked-TTL cap (MAX_TRACKED_TTL_SECONDS), so no watermark can still vouch for it. Not the 5-hour watermark lifetime: the cap is WATERMARK_TTL - MAX_FUTURE_BUFFER, and comparing against the larger number left an hour-wide band that served malformed entries |
envelope_expired |
Present in Redis but past the writer's own expiresAtMs. Both clients trust the writer's clock here, so hosts must be NTP-synced; there is no skew tolerance. |
age_exceeds_watermark |
A tracked entry of either framing that is itself older than the 4-hour tracked-TTL cap — the same invariant by age, against the same bound, and the only one that reaches the pickle path |
reversed_envelope_timestamps |
The envelope expires before it was created. Malformed rather than stale, and the lifetime guard cannot see it: the difference is negative, so > cap is false and it would pass as a plausible entry |
created_at_beyond_future_buffer |
A tracked entry stamped more than 1 hour (MAX_FUTURE_BUFFER_SECONDS) in the future. Staleness is watermark_ms >= created_at_ms, so a stamp beyond every reachable watermark can never be suppressed — the entry is immune to invalidation for its whole Redis TTL while every other guard passes. The bound is the exact frontier: an invalidation issued now writes a watermark of at most now + MAX_FUTURE_BUFFER. An hour of clock-skew tolerance falls out of it |
divergent_payload_encoding |
The payload holds a lone surrogate — as a character, or as the JSON escape \ud800 that ensure_ascii produces. Python keeps it, Go substitutes U+FFFD, and both report a hit, so the entry is refused on read rather than served as a silent disagreement. Refused on write too, by both encoders, so this can only be an entry from a foreign writer or an older build. Envelope.PICKLE is outside the rule entirely: Go cannot read a pickle entry at all, so nothing can disagree about one. The same label also covers a payload nested deeper than 500 levels — both clients refuse to inspect one, so it is refused rather than served unchecked; the log line names which of the two fired |
unloadable_payload |
The envelope parsed, but the Serializer could not load the payload — e.g. another language changed the payload schema |
unreadable_watermark |
The watermark is not a number at all. Suppresses the entry, then deletes the watermark so the next Python read recovers (the Go client cannot delete) |
non_finite_watermark |
The watermark is nan or inf. Suppresses every entry for that (key_type, id) until the watermark's own TTL expires |
A sustained nonzero rate on any of these means the entry is being rewritten on every read, which costs a fallback each time even though the answers stay correct.
Use both local and remote cache, rely on TTL:
GCacheKeyConfig(
ttl_sec={CacheLayer.LOCAL: 300, CacheLayer.REMOTE: 3600},
ramp={CacheLayer.LOCAL: 100, CacheLayer.REMOTE: 100},
)Good for: feature flags, configuration, rarely-changing data.
Use remote cache only (local can't be invalidated across instances), with invalidation:
# In config
GCacheKeyConfig(
ttl_sec={CacheLayer.LOCAL: 0, CacheLayer.REMOTE: 3600}, # No local cache
ramp={CacheLayer.LOCAL: 0, CacheLayer.REMOTE: 100},
)
# In your write path
async def update_user(user_id: str, data: dict):
await db.update_user(user_id, data)
await gcache.ainvalidate(key_type="user_id", id=user_id)Good for: user profiles, permissions, anything that needs immediate consistency.
Contributions are welcome! The project uses:
- pytest for testing (
pytest tests/) - ruff for formatting and linting
- mypy for type checking
- pre-commit for automated checks
# Setup
poetry install
# Run tests
pytest tests/
# Run all checks
pre-commit run --all-filesMIT License — see LICENSE for details.