Caching & Memory Tuning

patinaDB can keep a governed, RAM-budgeted cache of decoded graph objects above redb’s page cache. It is opt-in and off by default — with PATINADB_CACHE_LIMIT unset (or 0) nothing is cached beyond the OS/redb page cache and the pre-existing plan/statistics caches, at zero residual cost.

This chapter explains why the cache exists, the memory-budget model that keeps it from fighting the page cache, and how to size, observe, and tune it. For the raw knob table (env / flag / YAML) see Configuration → Cache memory budget; this chapter is the conceptual companion.

Status. All four RAM cache layers ship on main today: the memory-budget governor plus L1 decoded objects (vertices), L1 property values, L2 adjacency, and L3 query results — see the Layers section — all opt-in via PATINADB_CACHE_LIMIT (off by default, zero residual cost). A fifth, L4 disk-backed victim tier for expensive L3 results also ships, opt-in via PATINADB_L4_VICTIM_MAX_BYTES.

Why a cache above the page cache

redb — and, beneath it, the OS — already keep hot B-tree pages resident. That is level 0, and it is good: it avoids disk I/O. But L0 caches bytes, and it stops there. Every read still re-decodes those bytes: bincode / encode_for_index → Vertex / Edge / AttributeValue, with each string value heap-allocated multiple times on the decode → hydrate → pack path. On a hot OLTP path (point lookups, MATCH (n) WHERE n.id = …, fan-out target reads) that decode CPU is paid again and again for the same object.

The patinaDB cache sits above the byte boundary and caches the decoded artifact, so a hit skips the decode and the string allocations entirely. It never mmaps or manages pages itself — L0 stays the I/O-avoidance layer, and the cache is strictly a read-side accelerator: writes always go straight to redb through the durable path, and the affected cache entries are invalidated, never written through. A cache hit can only make a read faster (or memory tighter), never return a stale or wrong result.

The memory budget model

The app cache is a second consumer of the same RAM redb’s page cache needs. Grown naively it trades a decode-CPU win for extra page faults — a bad trade once the working set approaches RAM. So the budget is not a fixed number; it is an explicit, cgroup-aware partition that leaves the OS page cache a guaranteed floor.

How much do we have? (total discovery)

The number everything derives from is not the host’s MemTotal. In a container that is the host’s RAM, and trusting it is the classic Docker OOM-kill footgun (patinaDB ships in Docker). Discovery, in order:

1. cgroup limit — cgroup v2 memory.max, else v1 memory.limit_in_bytes (the real ceiling the kernel OOM-kills at);
2. host /proc/meminfo MemTotal (bare metal / unlimited cgroup);
3. TOTAL = min(cgroup_limit, MemTotal).

A non-Linux host, or unreadable files, falls back conservatively (better to under-budget than to over-commit against RAM we can’t measure).

The partition and the free-floor

From TOTAL patinaDB resolves its own-heap ceiling MEMORY_LIMIT (this does not include the page cache — that is OS-managed L0, protected separately by the floor below), then carves four regions:

PATINADB_MEMORY_LIMIT itself defaults to auto = TOTAL − min_free. The page-cache floor is expressed against system free RAM, not the internal MEMORY_LIMIT — it is the governor’s promise to the OS, orthogonal to the internal cache-vs-action split.

Each knob accepts an absolute size (8GiB, 512MiB), a fraction of its parent region (40%), or auto/unset for the default. Precedence per knob is explicit-absolute > fraction-of-parent > default, and fractions compose: a PATINADB_CACHE_LIMIT=40% is 40% of the resolved MEMORY_LIMIT, which may itself be a fraction of the discovered TOTAL. Every knob is also a patinadb-raft flag (--cache-limit, --memory-limit, --work-mem-limit, --mem-headroom, --cache-min-free) and a YAML config key, with the usual explicit-flag/env > file > default precedence — see the Configuration reference.

The elastic priority

Cache and query execution (“actions”) draw from the same heap and compete. Rather than a hard wall between them, the governor applies a priority with an elastic boundary:

under memory pressure:   actions  >  cache  >  (both yield to)  page-cache floor

A running query that needs working memory may evict cache to grow its action pool — a completing query beats a discardable cache — but the whole of MEMORY_LIMIT never pushes system-free RAM below PATINADB_CACHE_MIN_FREE. Two nested guards hold at all times: an internal one (cache + actions + headroom ≤ MEMORY_LIMIT) and an external one (MEMORY_LIMIT respects the page-cache floor).

Fail-loud validation

At startup patinaDB resolves the whole partition and refuses to boot if it over-commits — the errors name the offending knobs and the resolved byte counts:

• CACHE_LIMIT + WORK_MEM_LIMIT + HEADROOM ≤ MEMORY_LIMIT (internal partition);
• MEMORY_LIMIT + CACHE_MIN_FREE ≤ TOTAL (leave the OS its floor).

When caching is enabled the fully-resolved budget is logged once at startup and echoed on GET /mgmt/cache, so the sizing is never a mystery. The startup line looks like this (from a boot with PATINADB_CACHE_LIMIT=256MiB):

cache budget: memory_limit=… · cache=256.0MiB · work_mem=… · headroom=… · min_free=…

With caching off it instead logs cache: disabled (PATINADB_CACHE_LIMIT unset or 0).

A worked sizing example

Take a container with memory.max=16GiB and the defaults. TOTAL discovers as 16GiB; MEMORY_LIMIT=auto resolves to TOTAL − min_free (the floor keeps ≥1.6GiB of system RAM free so redb’s L0 breathes); the three internal regions then split it roughly Cache ≈ 5.8GiB · Actions ≈ 6.5GiB · Headroom ≈ 2.2GiB. Tune from there:

• Read-heavy / point-lookup workload — leave more RAM to the page cache: PATINADB_MEMORY_LIMIT=10GiB (≈6GiB stays with L0), and the decoded-object cache still captures the hot decode CPU.
• Feed / dashboard workload with heavy repeated reads — bias toward the cache: PATINADB_CACHE_LIMIT=70%, where repeat-read cache value dominates.

The governor

The budget is the contract; the governor is the runtime feedback loop that enforces it, “always keeping free RAM in view”:

• Observe. A lightweight background sampler reads MemAvailable (/proc/meminfo) on a 1-second interval — the ground truth of how much RAM is actually free right now, including pressure from other processes on the box.
• Protect the page cache first. When free RAM drops below PATINADB_CACHE_MIN_FREE, the governor shrinks the app cache by the deficit before the OS starts reclaiming the page-cache pages redb depends on. A hysteresis window keeps a steady stream of below-floor samples from re-evicting every tick (no thrash). Its standing bias is “shrink app caches first under pressure.”
• Admission = scan resistance. Admission uses W-TinyLFU (a small frequency sketch): a candidate is admitted only when it is estimated hotter than the entry it would evict. So a one-shot full analytics scan streams through without evicting the hot OLTP working set — a full table scan won’t flush the cache. Eviction within the budget is segmented LRU (probation → protected), weighted by value density = (hit-frequency × cost-avoided) / bytes, so a cheap-to-recompute big object yields before an expensive small one.
• Node-local, never stale. Caches are strictly node-local — nothing travels over Raft, and there is no cross-node coherence protocol. Correctness comes from generation tags: every write bumps the touched labels’ write-generation once at the sync() choke-point (the same mechanism behind fully_populated and the statistics catalog), and every cached entry carries the generation(s) it was built under. A lookup whose stamped generation no longer matches the live one is a miss + evict, never a stale hit — so a write to a label invalidates all of that label’s cached entries in O(1), with no scan and no per-key invalidation list. Time-travel (AS OF) reads and the copy-on-read write-resolve mirror both bypass the live cache.

What gets cached (the layers)

The design is a hierarchy above the page layer, extended incrementally:

Also folded under the governor’s accounting are the pre-existing plan cache and statistics / fully_populated catalogs.

The RAM layers share one budget + the generation-tag invalidation discipline; GET /mgmt/cache and CALL patinadb.cache.stats() report each level’s live bytes + hit rate. L4 is a disk tier with its own byte cap (not part of the RAM budget) — see below.

L4 victim cache (disk)

The L3 result cache is pure-RAM under the governor’s byte budget, and it discards a result in two cases regardless of how expensive it was: on eviction (the coldest entry is dropped when over cache_limit) and on admission rejection (a fresh result that loses the scan-resistance comparison). In both cases the expensive CPU/IO of computing the result is thrown away and the next identical query pays full price. Meanwhile the box usually has spare disk.

The L4 victim cache catches exactly those victims: when an L3 result whose measured compute cost exceeds a threshold is evicted, it is spilled to a separate redb file under the database directory (<db_root>/_l4_victim/), and a future identical read is served from disk (deserialize an already-materialized result) instead of recomputed. The read order becomes L3-RAM → L4 victim → recompute, and an L4 hit is promoted back into RAM.

• Provably as safe as L3-RAM. An L4 hit is validated against the same multi-label generation stamp L3-RAM uses — persisted beside the payload. A write to any involved label advances that label’s durable generation, so the stamp no longer matches and the entry is dropped on read (lazily) and recomputed. Never a stale hit.
• Survives a restart (unlike the pure-RAM tiers): the persisted key and stamp are db-id-free — db identity is the file location and validity rests on the per-label generations, which are durable in the main database. So an expensive dashboard query stays warm on disk across a node restart.
• Cost-gated. Only a result costing more than PATINADB_L4_VICTIM_MIN_COST_MS (default 50 ms) is spilled — a cheap query’s disk round-trip would cost more than just recomputing it.
• Bounded by its own cap with LRU eviction: PATINADB_L4_VICTIM_MAX_BYTES (0 = off, opt-in, the shipped default). L4 is not part of the RAM budget — it lives on disk and never competes with the OS page cache.
• Off the hot path. L4 is read only on an L3-RAM miss and written only on the (already-cold) eviction path, so no hot query takes an L4 lock. It inherits L3’s admission analysis verbatim (edge/traversal/unlabeled/procedure/non-deterministic shapes are never cached), is bypassed for time-travel (AS OF) and the write resolve mirror, and is truncated on clear_graph / snapshot install.

Enable it alongside the RAM cache (it catches RAM’s victims): e.g. PATINADB_CACHE_LIMIT=4GiB PATINADB_L4_VICTIM_MAX_BYTES=8GiB. Observability (a dedicated l4_victim metrics block + CALL patinadb.cache.stats row) ships alongside it — see Cache Observability & Tuning.

Observability & tuning

The cache exposes the same per-scope accounting four ways — CALL patinadb.cache.stats(), GET /mgmt/cache, the patinadb_cache_* Prometheus series (with a ready-made Grafana dashboard), and the patinadb-browser admin Cache section with its read-flow Sankey — so an operator can see which database, collection (label), or query shape is hot and how much RAM it holds.

The full treatment — every metric with its name on each surface, how to read each one, the Grafana panel walk, and a symptom → knob tuning playbook — lives in its own chapter: Cache Observability & Tuning. The short version: a high hit ratio on growing patinadb_cache_bytes means the cache is earning its keep (grow it); free-floor eviction with mem_available pinned at min_free means it is starving the OS page cache (shrink it).

When to disable. Set PATINADB_CACHE_LIMIT=0 (the default). If the working set comfortably fits RAM with the page cache alone and decode cost is already negligible, the app cache adds accounting overhead for little gain — L0 plus the plan/stats caches is the right baseline. Disabling is a fully supported, tested configuration with zero residual cost.

Honest limits

• Write-heavy scopes self-limit. A label under constant write churn bumps its generation constantly, so its cached entries rarely survive to a second hit — the cache naturally declines to cache churny data (its value density collapses) and spends the budget where it pays. The win is therefore workload-shaped: strong for read-heavy / feed / dashboard traffic, neutral for write-saturated.
• The biggest failure mode is oversizing the app cache and starving the OS page cache — inducing the exact page-cache eviction and swap cliff the cache was meant to avoid. The page-cache floor and the governor’s “shrink app caches first” bias exist precisely to prevent this, and the gauges above make a misconfiguration visible rather than silent. When in doubt, size the cache conservatively and let a high hit rate justify growing it.