Configuration Reference
A consolidated reference for the server’s knobs.
Cypher CASE depth
PATINADB_MAX_CASE_DEPTH bounds the total number of unquoted CASE keywords
in a statement, including independent sibling expressions. Its default is 200
in release builds and 56 in debug builds; positive overrides are clamped to
320 and 60 respectively. Zero or an invalid value selects the default. The
effective cap cannot exceed PATINADB_MAX_EXPR_DEPTH (default 200).
Exceeding the cap returns a syntax error before recursive parsing.
Debug builds give CASE-heavy parsing bounded extra stack on the caller’s thread (up to an 8 MiB temporary stack when needed). This preserves the existing limits and thread-local query scopes. On platforms where sufficient stack capacity cannot be established, these debug queries fail with an explicit capacity error. Release builds retain their existing parsing path; this is not a higher query-depth limit or a new configuration toggle.
Server (patinadb-raft)
| Flag | Default | Notes |
|---|---|---|
--config <path> | off | Load settings from a YAML config file (see below). |
--print-config-schema | off | Print a JSON Schema for the config file to stdout and exit. |
--check-config | off | Validate the resolved config, print what was understood, and exit without booting — an nginx -t-style dry run (see below). |
--id <u64> | required | Unique Raft node id (here or in the config file). |
--addr <host:port> | required | HTTP (REST + management + peer RPC). |
--db <dir> | required | Database root (one subdir per database). |
--bootstrap | off | Self-init a single-voter cluster. |
--join <member> | off | Auto-join an existing member as a learner on startup. Excludes --bootstrap. |
--bolt-addr <addr> | 127.0.0.1:7687 | Bolt listener; "" disables Bolt. |
--advertised-addr <a> | = --bolt-addr | Public Bolt address for routing behind a proxy. |
--auth-user <name> | neo4j | Auth username. |
--auth-password <p> | "" | Shared password. Empty = no auth, fail-closed: the node won’t start without --insecure-disable-auth. Use a long, random value (16+ characters) — issue #1065 exempts this exact username from the failed-login rate limiter below (a single-service-principal deployment must never be able to lock itself out of its own account), so unlike a directory-user password, no per-minute guess cap backs this one up. A short value logs a loud startup warning but does not refuse to boot. |
--insecure-disable-auth | off | Explicitly allow running open (empty password). Trusted networks only. |
--tls-cert <path> | off | PEM cert chain. With --tls-key, serves the HTTP plane (REST + peer RPCs) over HTTPS. |
--tls-key <path> | off | PEM private key (required with --tls-cert). |
--tls-ca <path> | system roots | PEM CA peers verify each other with (self-signed / private-CA clusters). |
--require-metrics-auth | on (default) | Gate GET /metrics behind Basic auth. Default-on since series carry db=<name> labels. |
--insecure-open-metrics | off | Opt out of the /metrics gate — serve it open on a private monitoring network. By default /metrics needs a credential and the global Admin role (issue #693: its db=<name> labels would otherwise let a per-db Reader enumerate other tenants). |
--query-timeout-secs <n> | 300 | Per-request budget for a REST /cypher read; overrun → 503 (deadline, not a hard cancel). 0 = off. |
--max-concurrent-requests <n> | 512 | Cap on in-flight client statements, node-wide — every REST request AND every Bolt statement (read, write, or inside an explicit transaction). Excess is shed with 503 on REST and a driver-retryable Neo.TransientError on Bolt. 0 = unlimited. |
--readiness-max-lag <n> | 50 | /ready apply-lag tolerance: report not ready (503 lagging) when last_log_index − last_applied exceeds this. See Day-2 operations. |
--max-leader-contact-age-secs <seconds> | unset (disabled) | Additional follower/learner freshness gate: require recent leader contact and bound last advertised leader-commit versus local-applied lag using readiness-max-lag. Applies to local read eligibility and readiness, not peer freshness inference. |
--otel-endpoint <url> (PATINADB_OTEL_ENDPOINT) | unset | Enable OTLP/HTTP diagnostic span export. Request correlation remains available without export; JSON logs are configured independently. See Day-2 operations. |
PATINADB_TRACE_SAMPLE_PERCENT | 1 | Integer 0–100: local trace-ID sampling percentage when OTLP is enabled. A caller’s sampled flag cannot override it. |
PATINADB_TRACE_SPANS_PER_SECOND | 128 | Integer 1–1024: node-wide admitted sampled-span budget per one-second window, not a request-rate limit. Excess spans are dropped. |
--allow-csv-dir <dir> | deny-all | Directory LOAD CSV FROM 'file://…' may read from. Repeatable. Unset ⇒ every LOAD CSV file read is refused. See Cypher file I/O. |
--allow-export-dir <dir> | deny-all | Directory the CSV export procs (patinadb.export.*) may write to. Repeatable. Unset ⇒ every export write is refused. |
--rbac-closed (PATINADB_RBAC_CLOSED) | off | Closed-mode RBAC: deny a non-admin any database it holds no explicit grant on. See Authentication & TLS. |
--telemetry-degrade-override-until <value> (PATINADB_TELEMETRY_DEGRADE_OVERRIDE) | unset | Time-boxed break-glass: keep writes enabled past a missed telemetry grace window until an absolute deadline. See Day-2 operations. |
--bookmark-wait-timeout-secs <n> (PATINADB_BOOKMARK_WAIT_TIMEOUT_SECS) | 10 | How long a Bolt BEGIN/RUN will block for this node to catch up to an incoming driver-session bookmark before failing with a retryable BookmarkTimeout. 0 disables the wait (bookmarks are parsed but ignored). It also bounds the initial leader/applied admission barrier for bulk upserts and the ordering/read-index wait for REST pipelined writes (/cypher autocommit and bulk), together with the remaining query timeout. Bolt pipelined writes are bounded only by the query timeout; lowering it can increase write admission refusals. 0 removes this admission bound but leaves the query timeout in force. See Bolt → Causal consistency and the write retry contract. |
Environment: PATINADB_AUTH_PASSWORD sets the auth password (preferred over a
shell flag).
Regional routing and scoped bookmarks
These optional YAML keys have matching kebab-case CLI flags and uppercase
PATINADB_ environment variables. Existing flag/environment-over-file precedence
applies. Defaults below are resource/rollout policy, not measured latency promises.
| YAML key | Default | Meaning |
|---|---|---|
causal_bookmark_mode | legacy | legacy, compatible, or strict; see Bolt. |
regional_routing_enabled | false | Requires compatible/strict mode and positive bookmark timeout. |
node_name | unset | Advisory display label, 1–128 UTF-8 bytes, no controls. Numeric node ID remains authoritative. |
node_region | unset | 1–64 lowercase ASCII letters/digits/_/-, starting with a letter or digit. |
regional_routing_table_ttl_secs | 30 | Table TTL cap, 1–300 seconds; actual TTL may be zero. |
regional_routing_observation_max_age_secs | 15 | Fresh-profile lifetime bound, 1–300 seconds. |
regional_routing_coordination_timeout_ms | 3000 | Total coordination deadline, 1–30000 milliseconds. |
regional_routing_max_concurrent | 16 | Separate request/profile admission caps, 1–1024. |
See Multi-Region Topology for routing URIs, reader tiers, failure behavior and the all-members-upgraded prerequisite.
More server flags (RBAC, Bolt/transaction limits, telemetry, ops)
The table above covers the flags you’ll set on day one. These round out the
full surface — most have safe defaults and only need touching for a specific
requirement (multi-tenant RBAC, transaction-buffer bounds, dead-voter
auto-repair, telemetry/licensing). Every one is also settable via
PATINADB_* env var and/or the YAML config file, same precedence rules.
| Flag | Default | Notes |
|---|---|---|
--cluster-secret <s> (PATINADB_CLUSTER_SECRET) | falls back to --auth-password | Dedicated peer /raft/* secret, separate from the root-admin credential so the two rotate independently. Required for clustered nodes: with auth enabled, a non-bootstrap node refuses to start without a dedicated value (a single --bootstrap node may omit it, with a warning). All nodes of a cluster must share the same value. |
--rbac-closed (PATINADB_RBAC_CLOSED) | off | Deny a non-admin any database it holds no explicit grant on (tenant isolation). See Authentication & TLS. |
--rbac-rel-grants (PATINADB_RBAC_REL_GRANTS) | off | Strict relationship-type RBAC: a label-scoped user must also hold a per-rel-type grant for every edge type traversed. See Authentication & TLS. |
--rbac-rel-property-grants (PATINADB_RBAC_REL_PROPERTY_GRANTS) | off | Strict relationship-property RBAC: a label-scoped user must also hold a per-(relType, prop) grant. See Authentication & TLS. |
--provenance-required (PATINADB_PROVENANCE_REQUIRED) | off | Refuse (409, before proposing) a write whose provenance/tx_metadata context is non-empty but Anamnesis is off for the target db, instead of silently dropping the context (issue #1071). REST autocommit only today — Bolt writes are not yet covered. See Anamnesis. |
--auth-max-attempts-per-min <n> (PATINADB_AUTH_MAX_ATTEMPTS_PER_MIN) | 20 | Brute-force auth throttle, per source IP and per username, for directory (CREATE USER) accounts. Blocks a failed-login flood before it reaches the argon2 verify. Set 0 to disable (startup logs a warning). Does not apply to the root-admin username (--auth-user/--auth-password) — issue #1065 exempts it entirely; see --auth-password above. |
--auth-verify-cache-ttl-secs <n> (PATINADB_AUTH_VERIFY_CACHE_TTL_SECS) | 60 | How long a directory account’s successful argon2 verify is cached, so a repeated correct password skips argon2 on the next request (issue #1067 — REST re-authenticates every request, so without this a busy directory account pays a ~19 MiB argon2 verify per request). Set 0 to disable. A security trade-off, not a tuning knob: bounds how long a revoked credential could still authenticate if a change ever reached the node outside UserStore’s own methods — today nothing does, since ALTER USER … SET PASSWORD/DROP USER/grant changes invalidate synchronously regardless of this value, so the practical window is zero. See Authentication & TLS. |
--max-reads-per-user <n> (PATINADB_MAX_READS_PER_USER) | 0 (unlimited) | Cap concurrent reads per authenticated principal, on both surfaces — one user’s burst of expensive read procs can’t exhaust the blocking pool. |
--max-writes-per-user <n> (PATINADB_MAX_WRITES_PER_USER) | 0 (unlimited) | Cap concurrent writes per authenticated principal, on both surfaces. Not redundant with the per-database write lock: with pipelined writes (on by default) a write’s resolve runs off that lock on the blocking pool, so one principal can otherwise hold as many concurrent resolves as --max-concurrent-requests allows. Earns its keep on a multi-tenant node (--rbac-closed); with a single trusted backend it bounds the same set as the node-wide cap. |
--blocking-threads <n> (PATINADB_BLOCKING_THREADS) | tokio default (512) | Absolute ceiling on the tokio blocking-thread pool. |
--max-bolt-connections <n> | 1024 | Cap on concurrent authenticated Bolt connections; 0 = unlimited. An unauthenticated connection never spends this budget (see the note below, issues #440/#730). |
--bolt-handshake-timeout-secs <n> (PATINADB_BOLT_HANDSHAKE_TIMEOUT_SECS) | 10 | Close a Bolt socket that sends nothing (or stalls mid-handshake) before completing the version-negotiation handshake. 0 disables. |
--bolt-idle-timeout-secs <n> (PATINADB_BOLT_IDLE_TIMEOUT_SECS) | 300 | Close a Bolt connection that sits with no open transaction and no started result stream and sends no message for this long (covers a connection that finishes HELLO/LOGON then goes silent). 0 disables. |
--bolt-auth-timeout-secs <n> (PATINADB_BOLT_AUTH_TIMEOUT_SECS) | 30 | Close a Bolt connection that has not successfully authenticated within this long — a hard deadline, armed once at connection start, that no client message can reset (unlike --bolt-idle-timeout-secs above). See the note below (issue #730). 0 disables. |
--idle-tx-timeout-secs <n> (PATINADB_IDLE_TX_TIMEOUT_SECS) | 300 | Roll back + close an explicit Bolt transaction left idle this long (“idle in transaction” guard). 0 disables. |
--max-tx-ops <n> (PATINADB_MAX_TX_OPS) | 5,000,000 | Max buffered DeltaOps an explicit Bolt transaction may accumulate before it’s aborted. 0 = unlimited. |
--stream-pull-timeout-secs <n> (PATINADB_STREAM_PULL_TIMEOUT_SECS) | 300 | Cancel a streaming Bolt RUN result the client never PULLs within this window. 0 disables. |
--audit-max-entries <n> (PATINADB_AUDIT_MAX_ENTRIES) | 100,000 | Retention for the durable node-local security-audit log. Since issue #1069 this bounds only ROUTINE entries — every denial + ddl.user/ddl.admin/rotate-*/retire-* is exempt (see --audit-max-protected-entries). 0 = unlimited. |
--audit-max-protected-entries <n> (PATINADB_AUDIT_MAX_PROTECTED_ENTRIES) | 1,000,000 | A SEPARATE hard safety-net cap on PROTECTED entries only (issue #1069 review follow-up — “protected means long-lived, not immortal”). Measured ~0.39 GiB at the default; see Authentication & TLS. 0 = unlimited (explicit opt-out). |
--auto-evict-after-secs <n> (PATINADB_AUTO_EVICT_AFTER_SECS) | 0 (disabled) | Auto-evict a voter that stays dead (unreachable + not replicating) past this many seconds, using the same quorum-preserving guard as POST /mgmt/evict-voter. See Day-2 operations. |
--license <path|token> (PATINADB_LICENSE) | none (community) | Signed license file or inline token. See Licensing & Telemetry. |
--install-name <text> (PATINADB_INSTALL_NAME) | unset (anonymous) | Opt-in, operator-chosen telemetry label. |
--telemetry-interval-secs <n> | 21600 (6h) | Heartbeat interval. |
--telemetry-grace-secs <n> | 259200 (72h) | Community-mode grace window before write-degrade. |
--disable-telemetry | off | Turn off telemetry entirely — requires a valid license (fail-closed otherwise). |
Also note the unauthenticated-Bolt-connection lifecycle (issues #440/#730):
--max-bolt-connections bounds only authenticated sessions — its permit is
acquired at the instant a connection first authenticates
(BoltConn::admit_connection), never at TCP accept, precisely so a flood of
connections that never authenticate can’t starve real clients of it. Every
phase BEFORE that point is instead bounded in time, end to end, with no gap:
--bolt-handshake-timeout-secs covers accept → the Bolt version handshake, and
--bolt-auth-timeout-secs covers the version handshake → first successful
authentication — the latter closes a residual the (message-resettable)
--bolt-idle-timeout-secs cannot: a connection that never authenticates but
keeps itself alive forever with cheap pre-auth traffic (a bare HELLO, or any
message answered without auth) that repeatedly resets the idle clock. Unlike
the idle-read timeout, the auth deadline is armed once as a fixed instant and
is never reset by any client message, of any kind or cadence.
Also note PATINADB_MAX_CONCURRENT_RAFT_RPCS — a separate, env-var-only
backpressure cap (default 1024, 0 = unlimited) on inbound peer /raft/*
RPCs, distinct from --max-concurrent-requests (which governs client
traffic) so a peer-RPC flood and a client-request flood can’t shed each
other’s capacity.
Raft timing and the round-trip envelope
Four env-only knobs set this node’s Raft timing, plus one stall bound for snapshot transfers (issue #1068). Leaving them unset gives the LAN-calibrated defaults the server has always used, so an unset environment behaves exactly as before.
| Variable | Default | What it bounds |
|---|---|---|
PATINADB_RAFT_HEARTBEAT_MS | 250 | Heartbeat period and the timeout on every append_entries RPC |
PATINADB_RAFT_ELECTION_TIMEOUT_MIN_MS | 750 | Election timeout, lower bound |
PATINADB_RAFT_ELECTION_TIMEOUT_MAX_MS | 1500 | Election timeout, upper bound |
PATINADB_RAFT_INSTALL_SNAPSHOT_TIMEOUT_MS | 300000 | The snapshot resume-probe and the post-upload install/response wait — since issue #1068 it no longer caps the streamed upload itself |
PATINADB_RAFT_SNAPSHOT_PROGRESS_TIMEOUT_MS | 30000 | Abort a snapshot upload after this long with zero bytes flowing; total transfer duration is unbounded while bytes keep flowing (issue #1068) |
Give every node in a cluster the same values. They encode a shared assumption about the network, not a per-node preference.
The first one decides which peers can replicate at all. It is not only the
heartbeat period: every append_entries — payload as well as heartbeat — is
abandoned if it does not complete within it. That one number is therefore the
whole budget for request upload plus round-trip delay plus the follower’s
log fsync plus the response. Small heartbeat packets fitting the budget do
not prove that a bulk-write payload or catch-up batch will fit. Account for the
encoded body size and available throughput, and tune the entry/body bounds below
together with the timing envelope; test both steady writes and catch-up.
A peer that cannot answer inside the budget never acknowledges an
append, so the leader’s matched index for it never advances and nothing
commits through it. This applies to learners exactly as it does to voters.
This failure is silent, and that is the dangerous part. Followers reset
their election timer when a heartbeat arrives — one way, far inside the
election window — so no election storm draws attention to it. /mgmt/metrics
reports a stable leader and healthy members. Writes that need the distant peer
simply never commit, and a learner behind such a link stays empty indefinitely.
If you are debugging “the cluster looks fine but writes to it hang”, check this
budget against your real round-trip times first.
Measured under issue #1066 on a 3-node cluster with both follower links delayed (injected one-way delay against the default 250 ms budget):
| Injected delay | Writes committed |
|---|---|
| 100 ms | 3 / 3 |
| 200 ms | 3 / 3 |
| 300 ms | 0 / 3 |
It is a cliff, not a slope — nothing degrades between 200 ms and 300 ms, it
stops. Raising only PATINADB_RAFT_HEARTBEAT_MS to 2000 ms made the identical
cluster commit 3/3 again at the same 300 ms delay, which is what identifies the
budget as the cause rather than latency in general.
Two caveats on those numbers. They come from latency injected into the peer
transport, not from a kernel netem link: the request leg is delayed, with no
jitter, reordering or packet loss, so treat them as the budget boundary rather
than as a model of a real WAN. And raising the budget is not free — it buys
reach by making every failure detection correspondingly slower, so raise the
election timeouts with it (keep the election window above roughly three times
the heartbeat, the standard Raft guidance; --config-check complains if you
do not).
Rough guidance, given a 250 ms default: a same-continent link (on the order of 80–120 ms round trip) fits comfortably. An intercontinental one (roughly 200–350 ms) sits on or beyond the edge and needs a raised budget — and every write then costs that round trip, so consider whether a stretched voter set is what you want, or whether a local read replica per region serves you better.
And PATINADB_SNAPSHOT_LOGS_SINCE_LAST (env-only, default 5000, 0
or an unparseable value = the default) — how many applied Raft log entries
trigger a snapshot build on a node. The default is openraft’s own, so leaving
it unset changes nothing.
Worth relating to your write rate, because the two halves are set by different things: how often a snapshot is built follows your traffic, while how much it costs follows your database. At 10 writes/s the default fires roughly every 500 seconds — about 173 builds per node per day — and each build streams that node’s graph. Raising the value trades a longer Raft log (more log disk, and a follower that falls far behind has more entries to replay before it catches up) for fewer builds; lowering it does the reverse. Measure a build on your own data before changing it: issue #1099 measured 0.77 s for a 60 000-vertex graph on NVMe, but that number is about that graph, not yours.
And PATINADB_PEER_RPC_ENCODING (env-only, json default) — the
outbound peer-RPC transport encoding. Every node’s /raft/append and
/raft/vote receiver accepts BOTH legacy JSON and the framed-postcard
encoding (sniffed per request from Content-Type, answered in the request’s
format), so opting any subset of nodes into postcard can never split the
cluster. postcard switches this node’s sender to the compact
framed-postcard encoding; the JSON default is unchanged (a project-wide
default flip is deferred — patinaDB is not live in production). Receive-side
format mix is observable via
patinadb_raft_peer_rpc_received_total{rpc,format}.
And PATINADB_MAX_SNAPSHOT_BYTES (env-only, default 8 GiB, 0 =
unlimited) — the receive cap on the streamed inter-node snapshot upload
(POST /raft/snapshot). Since issue #1246 it windows a single upload
attempt: over the cap the attempt aborts with 413 but the partial file
is KEPT, and the sender’s probe/resume chain continues from that offset — so
a snapshot larger than the cap (a 60M-element fixture produces a ~19.5 GiB
compressed snapshot, measured in the #1145 rejoin drill) installs over
several attempts instead of never. It exists because axum’s
DefaultBodyLimit is a no-op on that streamed route. Watch
patinadb_raft_snapshot_receive_rejected_total{reason="over_cap"}: a counter
that climbs without the follower catching up means the chain is not
converging. See
High Availability.
And the replicated-entry size bounds (issue #839), both env-only:
PATINADB_MAX_ENTRY_BYTES(default 48 MiB,0= disabled — NOT recommended) — the proposer-side cap on a single Raft entry’s peer-wire size. A statement whose resolved write would frame anAppendEntriesRequestbody over the followers’/raft/appendreceiver cap is rejected before anything is proposed, with a non-retryable client error (Neo.ClientError.Statement.ArgumentError/ HTTP400) naming the limit — an oversized entry that DID reach the wire would be413’d by every follower, retried forever by openraft, and would wedge the cluster into elections. If you hit this limit, split the write into smaller batches:CALL { … } IN TRANSACTIONS OF <n> ROWSfor Cypher, smallerUNWINDbatches from the driver, orchunk_rowson the bulk-upsert endpoints.PATINADB_RAFT_MAX_BODY_BYTES(default 64 MiB,0/unparsable = default; never unbounded) — the receiver-side body cap on the peer/raft/append//raft/vote//raft/snapshotendpoints. Raising it gives bulk workloads more headroom, but it must be set identically on every node (a follower with a lower cap than the leader’s entries413s exactly as before), andPATINADB_MAX_ENTRY_BYTESshould be raised with it, staying comfortably under this value. This knob is also what the sender budgets its append bodies against, so that a batch of individually-legal entries cannot exceed the cap when a follower is catching up (see Batched appends) — on a link slower than ~1.9 Gbps, lower it so one budgeted body still crosses the wire inside Raft’s 250 ms RPC deadline.
Admission counts the resolved request in both supported encodings (Postcard and
JSON), retaining no complete output body for sizing, and stops when either
exceeds the entry cap. JSON escaping and structural overhead count toward it;
a small Postcard representation is not enough to pass. The check runs after
request construction and does not limit transaction heap, serializer-owned
scratch or the buffers used for actual persistence/transport. Leave room for the
Raft entry and RPC envelope above the request; equal entry/body settings provide
no envelope headroom. The 48/64 MiB defaults and explicit 0 behavior above
are unchanged.
The two are an ordered pair, not two independent dials:
PATINADB_MAX_ENTRY_BYTES ≤ PATINADB_RAFT_MAX_BODY_BYTES. Above the body
cap the entry cap cannot protect anything — the guard waves through an entry
every follower will 413. --check-config fails (exit 1) on a misordered
pair; a node that boots with one warns loudly and clamps its effective entry
cap down to the body cap rather than refusing to start.
YAML config file (--config)
Instead of (or alongside) flags, point the server at a YAML file mirroring the settings above:
# node.yaml
id: 1
addr: "127.0.0.1:21001"
db: "/var/lib/patinadb"
bootstrap: true
bolt_addr: "0.0.0.0:7687"
auth_user: "neo4j"
# auth_password: prefer the PATINADB_AUTH_PASSWORD env var over the file
query_timeout_secs: 30
max_concurrent_requests: 256
patinadb-raft --config node.yaml
Every field is optional; an absent key falls back to the CLI flag, then to the
built-in default. The YAML key names match the long flag names with -
replaced by _ (e.g. --bolt-addr → bolt_addr). Unknown keys are rejected so
a typo fails loudly.
Precedence (highest wins): an explicitly-passed CLI flag (or its bound env
var, e.g. PATINADB_AUTH_PASSWORD) > the config file > the built-in
default. A flag left at its clap default does not override a value set in the
file — only flags the operator actually typed do. The required settings (id,
addr, db) may come from either the file or flags; if neither supplies one,
startup fails with a clear error.
Config JSON Schema (--print-config-schema)
patinadb-raft --print-config-schema prints a JSON Schema (Draft 7) for the
config file — every property carries its description (lifted from the Rust
doc-comments) so editors can offer autocompletion and validation. It works
without any other arguments:
patinadb-raft --print-config-schema > patinadb-config.schema.json
Config dry run (--check-config)
patinadb-raft --check-config is an nginx -t for patinaDB: it resolves the
effective configuration exactly the way a real boot does — CLI/env >
<VAR>_FILE/_COMMAND > --config file > default — prints what it
understood, validates it, and exits without starting the node. Exit code
0 means valid, 1 means a setting is invalid or a runtime env knob is set to a
value the engine would silently ignore. Drop it into a deploy script or CI gate:
patinadb-raft --id 1 --addr 127.0.0.1:21001 --db /var/lib/patinadb \
--bootstrap --auth-password "$PW" --cache-limit 40% --check-config || exit 1
It prints the resolved node identity, cluster mode, security posture (flagging auth-on-without-TLS), every limit with its effective value, the resolved cache budget in bytes (so you can see the caches are actually enabled and with how much), the telemetry/licensing state, and the filesystem sandbox.
Its most valuable job is the runtime env-knob scan. Several engine knobs
(PATINADB_MAX_AGG_ROWS, PATINADB_CACHE_LIMIT, …) are read lazily deep in the
engine and fall back to their default if the value doesn’t parse — silently.
So PATINADB_MAX_AGG_ROWS=5M looks like five million but is ignored (the runtime
wants a plain integer, 5000000), and — the case that motivated this feature — a
PATINADB_CACHE_LIMIT typo could silently leave the cache off. --check-config
resolves each set knob with the same parser the runtime uses and fails the
check on any set-but-unparseable value, naming it:
Runtime env knobs (curated — set values only)
PATINADB_CACHE_LIMIT 512MiB → 536870912 bytes
PATINADB_MAX_AGG_ROWS 5M ✗ expected a plain integer, got '5M'
configuration INVALID — 1 problem(s):
✗ PATINADB_MAX_AGG_ROWS is set to '5M' but expected a plain integer … SILENTLY IGNORED
The four RFC 0042 query-engine switches are on by default, so the dry run prints
their effective modes even when none is set, marking defaults and auto modes that
are inert because the cost model is off. A set value outside a switch’s grammar is
flagged too; for these switches the runtime selects off, not the default:
Query engine switches (RFC 0042; read once per process)
PATINADB_COST_MODEL off (set)
PATINADB_FACTORIZE on (default)
PATINADB_WCOJ off (set)
PATINADB_CSR auto (default, inert: the cost model is off)
It also cross-checks the peer-wire size caps as an ordered pair (each value
parses fine on its own; only the comparison catches a misordering) — see
PATINADB_MAX_ENTRY_BYTES above.
On/off knobs: two different grammars
Flag-shaped knobs come in two kinds, and the dry run now says which one you are looking at, because they behave oppositely for the same value:
-
Boolean knobs —
PATINADB_RAFT_GROUP_APPLY,PATINADB_PIPELINED_WRITES,PATINADB_PARTIAL_TIMETRAVEL,PATINADB_INDEXED_SNAPSHOTS,PATINADB_IMPORT_SORTMERGE_PARALLEL,PATINADB_PLAN_CACHE,PATINADB_SYNC_SNAPSHOT.0,false,off,noturn them off (case-insensitive);1/true/on/yesturn them on; unset — or set to an empty value — means the knob’s own default. The dry run renders the verdict (false → off). -
Negated boolean knobs —
PATINADB_NO_PARSE_CACHE,PATINADB_DISABLE_COMPRESSION. Same grammar, but read the name: a truthy value turns the named negative on, i.e. it disables the feature (PATINADB_NO_PARSE_CACHE=onmeans “no parse cache”). A falsy value, an empty value, or leaving it unset all leave the feature enabled. These are still ordinary booleans — unlike the presence switches below,=falsedoes the sensible thing. -
Presence switches — every
PATINADB_DISABLE_*kill switch, plusPATINADB_TRAVERSAL_LIVENESS_PROBE. These are presence-based: any value turns them on, including0andfalse. To turn one off you must unset the variable, not assign it a falsy value. The dry run spells this out next to the value rather than pretending it is a boolean:PATINADB_DISABLE_OBJECT_CACHE false → ON (presence-based: any value, INCLUDING '0'/'false', enables it — unset the variable to turn it off)PATINADB_TRAVERSAL_LIVENESS_PROBEis read once per server process, at the first match iterator, and inherited by every traversal step. Changing it requires a restart (as for any process environment variable).If your deployment tooling emits
VAR=falsefor a disabled feature, that will enable these — omit the variable instead.
Secrets management
The three sensitive server settings — PATINADB_AUTH_PASSWORD (root-admin
credential), PATINADB_CLUSTER_SECRET (peer /raft/* shared secret), and
PATINADB_LICENSE — can be sourced without ever putting the plaintext value in a
-e env var or a config file. patinaDB embeds no secrets-manager SDK; two
conventions integrate one while keeping its client out of the binary (the server
only ever sees the resolved value at startup):
-
<VAR>_FILE— names a file whose contents are the real value (mirrorsPOSTGRES_PASSWORD_FILE). Point it at a Docker Composesecrets:file, a KubernetesSecretvolume, or a tmpfs file a Vault Agent / AWS or GCP Secrets Store CSI-driver sidecar keeps refreshed — the DB reads the file, the sidecar owns the fetch + rotation. This is the pattern to prefer when you need auto-rotation. A trailing newline is trimmed. -
<VAR>_COMMAND— names a shell command whose stdout (trailing newline trimmed) is the value, for a direct dynamic fetch with no sidecar:PATINADB_AUTH_PASSWORD_COMMAND="vault kv get -field=pw secret/patinadb" # AWS: "aws secretsmanager get-secret-value --secret-id patinadb/pw \ # --query SecretString --output text" # GCP: "gcloud secrets versions access latest --secret=patinadb-pw"The command runs via
sh -c "<cmd>"once at startup (the fetch CLI must be present + authenticated). Fail-closed: a command that can’t spawn, exits non-zero, or produces empty output aborts startup with a loud error — it never silently falls back to “no secret” (which would look like--insecure-disable-auth). The fetched value is never logged.
Precedence (most specific wins):
direct flag/env > <VAR>_FILE > <VAR>_COMMAND > config file >
built-in default. Both _FILE and _COMMAND are only consulted when the direct
flag/env is unset. See deploy/README.md for the sidecar + command examples.
Resource limits & quotas
Guards that bound the cost of a single query so one statement can’t exhaust
memory or run unbounded. The engine budgets are environment variables; the
two server request limits are flags (see the patinadb-raft table above).
| Limit | Where | Default | Purpose |
|---|---|---|---|
PATINADB_MAX_HOPS | env | 1000 | Depth cap for an unbounded variable-length hop ([*], or [*..n] with n unset). An explicit [*a..b] in the query always wins. Prevents runaway traversal on a cyclic graph. |
PATINADB_CARTESIAN_CAP | env | 10000 | Max rows a disjoint (cross-product) multi-MATCH may produce before the query errors. Stops an accidental N×M blow-up. |
PATINADB_MAX_AGG_ROWS | env | 5000000 | Cap on the O(input)/O(result) row buffers behind GROUP BY/aggregate, a full (non-top-K) ORDER BY, UNION dedup, and a hash-join build side. A clear error over the cap instead of a silent OOM. See Query Planning. |
PATINADB_MAX_CAPTURE_OPS | env | 5000000 | Cap on the number of resolved ops a single write statement (the Raft leader’s resolve step) may buffer before recording one engram/Raft entry. A whole-graph SET/bulk CREATE over the cap fails with a clear error pointing at CALL {…} IN TRANSACTIONS instead of risking an OOM or a giant single Raft entry. |
PATINADB_MAX_UPSERT_ELEMENTS | env | 5000000 | Cap on nodes + edges a single JSON subgraph upsert (POST /mgmt/upsert-graph, RFC-0020) may resolve — checked before the O(n) resolution structures (by_label, per-label Vec<UpsertRow>, the resolved DeltaOp batch) are allocated, so an over-cap request fails in O(1) instead of amplifying the request body into many times its wire size. Admin-only endpoint (defense-in-depth, not a remote/unauth DoS path). Over the cap: split the load with Dataset::upsert_subgraph_chunked / the REST request’s chunk_rows field. |
PATINADB_RESTORE_OPS_CHUNK | env | 5000 | Operations per replicated entry when a restore (CALL patinadb.restore('<id>') or POST /mgmt/restore) proposes its delta. A restore larger than one chunk commits as several entries and is therefore not atomic — intermediate partially-restored states are observable. 0/unparsable falls back to the default (it does not disable chunking). See Engrams. |
PATINADB_MAX_ALGO_WORK | env | 1e9 | Static work-budget backstop for O(V·E) read procedures (betweenness/closeness): refuses a call whose estimated n·(n+e) exceeds this, so a Reader can’t trigger unbounded compute even without --query-timeout-secs. Admits realistic analytic graphs (10k nodes / 50k edges ≈ 6e8); raise it for a genuinely large centrality run. See Day-2 operations. |
PATINADB_WASM_MAX_DECODE_DEPTH | env | 256 | Nesting cap for the result blob a WASM procedure returns (see WASM procedures). The host decoder is recursive and runs after the guest call returns, so fuel/epoch/max_wasm_stack no longer bound it; a hostile guest could emit thousands of nested one-element lists in a few bytes and overflow the host stack. Over the cap the call fails with a clean procedure error. A real proc result is 1–3 levels deep. |
PATINADB_MAX_SNAPSHOTS | env | unlimited (0/unset) | Prunes on-disk periodic time-travel snapshot files down to the N most recent (+ every pinned/tagged one) after each snapshot-taking commit. A long-lived, write-heavy database otherwise grows snapshot files unbounded; pruning only slows reconstruction of an old, out-of-window engram — every AS OF result stays byte-identical. The reference deploy/ compose stack sets this to 16 (the code default stays unlimited). |
Write pipelining
| Knob | Default | Purpose |
|---|---|---|
PATINADB_PIPELINED_WRITES | on | Lets concurrent autocommit writes to one database resolve in parallel instead of serializing behind each other’s whole Raft round-trip, admitting each under a short optimistic conflict check. =0 is the rollback valve — writes then serialize per database, as before. |
Pipelining is what makes the optimistic concurrency model in
Production Deployment the one
you actually get: with it on, a write–write conflict between concurrent writers
is detected and reported as the driver-retryable
Neo.TransientError.General.TransientError rather than being avoided by
serializing everyone. There are no lost updates either way — a client (or,
better, a driver’s managed transaction) simply retries the transient.
The trade-off is isolation: pipelined autocommit writes run at snapshot isolation, the same level Bolt’s explicit transactions already use, so write skew is possible — two writers each reading a value and writing disjoint rows based on it leave no write–write overlap to detect. That matters only for a multi-row invariant that genuinely needs serializability; the sentinel-node pattern in Concurrency & retries turns such a case back into a detectable conflict. Turning the knob off restores serialized execution per database at a real throughput cost (measured ~4–5× on a disjoint-write workload).
Query parallelism (scan + aggregation)
Vector-index training has a separate shared pool:
PATINADB_VECTOR_BUILD_THREADS accepts positive integers capped at 8 (1 is
serial). The default is available logical CPUs minus one, bounded to 1–8;
invalid values use that default. It is read on first training use, so restart
after changing it. --check-config rejects invalid values (including zero)
and reports the capped worker count. This does not parallelize redb writes
or set a RAM budget.
See Vector Search.
PATINADB_FACTORIZE also gates the RFC 0042 B3–B5 factor consumers:
top-k within a factor, DISTINCT/existence over one factor, and cross-branch
inclusion–exclusion counts. See
query planning.
The RFC 0042 switches at a glance; details in the table below and in How the Query Planner Works.
| Knob | Default | Purpose |
|---|---|---|
PATINADB_WCOJ | auto | Closing probes, sorted neighbour intersection/anti-intersection and base-adjacency cyclic joins. Values: exactly off, on or auto (case-insensitive). Unset or empty means auto; any other value, including Boolean aliases, means off. Read once per process. off restores the reference traversal. auto requires PATINADB_COST_MODEL (entry or join), otherwise it acts as off: closing probes need a cached degree, and intersections are admitted per candidate only when bounded domain observations predict a cost advantage; missing evidence keeps binary traversal. Anti-intersection and cyclic joins remain explicit-on only. Cyclic joins support up to sixteen vertex variables and retain physical relationship bags, uniqueness and label checks. Native query memory admission is required; refusal replays binary execution before output. Unsupported shapes retain binary traversal. Default auto since 2026-10-17 (RFC 0042 C0–C2, A4; #1442). |
PATINADB_FACTORIZE | on | Factorized execution (RFC 0042 B2–B5): factor-local grouped aggregation, top-k within a factor, DISTINCT/existence over one factor and cross-branch inclusion–exclusion counts. Boolean grammar: unset or empty means on, 0/false/off/no mean off (restoring the reference operators); read once per process. Requires a query memory scope and proven independent fixed branches, or branches whose shared relationship types the runtime witness proof covers. Unsupported shapes and value domains retain the reference operator. With PATINADB_COST_MODEL=off every proven shape engages. With a cost model only the B2 per-pivot count path (count(*), count(x) and count(DISTINCT x) of the pivot or a branch end) is cost-admitted; other B2 shapes (sum, min, max, joined patterns) keep the flat aggregate, and the B3–B5 consumers have no cost admission. Default on since 2026-10-17 (#1442). |
PATINADB_COST_MODEL | join | Cost model: off, entry or join (see below). Unset or empty means join; off and every other value mean off, the reference planner. Any value other than off also restricts PATINADB_FACTORIZE grouped aggregation (B2) to the per-pivot count path, engaged only when the cached degree catalog (CALL patinadb.stats.degrees()) predicts that it costs less than the flat aggregate; missing or refresh-due degree evidence and other B2 shapes keep the flat aggregate. The B3–B5 factor consumers are not costed. Read once per process. PROFILE records the estimates under agg.factorized. |
PATINADB_CSR | auto | In-memory adjacency cache for single-type fixed-length MATCH, variable-length traversal and shortest paths. Accepts off, on, auto and Boolean aliases. Unset or empty means auto; any other value means off, the reference reader. auto requires PATINADB_COST_MODEL (entry or join), otherwise it acts as off; it admits only deduplicating variable-length traversals whose estimated expansions amortize the build; fixed-length MATCH, shortest paths and PATINADB_WCOJ intersections retain the reference reader. Missing degree data requests bounded background statistics. Builds asynchronously; unbuilt, dirty or over-budget data uses the existing reader. Takes one ninth of the entry-cache allowance (under auto, only once a traversal was admitted); the existing levels share the remainder. PROFILE reports traversal.source. Performance is workload-dependent and not generally qualified; default auto since 2026-10-17 (#1442). No persisted format change. |
PATINADB_QUERY_THREADS | 1 (off) | Per-query execution-thread cap, including the caller, clamped to 1–32. Values above 1 enable eligible read-only scans, fixed traversals, mergeable aggregates, bounded top-K with SKIP, and seeded MATCH Apply over ordered outer rows. Applicable serial path-count, grouped path-count, covering-input and deferred-prefix specializations retain priority at every thread cap. Results and ordering retain the serial contract. Secondary and multiple required entry labels are supported when the inverse index is ready. sum/avg and unsupported shapes remain serial; merge enforces the usual aggregate caps. |
PATINADB_QUERY_POOL_THREADS | Available parallelism, clamped to 1–32 | Maximum persistent background query workers per process, independent of Tokio and vector training. Application caller threads are additional. Read once when initialized. |
PATINADB_QUERY_MORSELS_PER_WORKER | 1 | Target scan morsel count per primary/secondary segment is this factor times the per-query worker cap; factor clamped to 1–64. A 1024-row sizing heuristic reduces partitions on small inputs. Plain scans use UUID bounds; traversal drivers use key-count sampling. Edge-range partition limits are unchanged. Tune against the workload. |
PATINADB_PAR_AGG_MIN_ROWS | 100000 | Minimum estimated label cardinality for the existing scan executors; top-K/Apply admission instead estimates reachable traversal work from outer rows and bounded fan-out samples. With PATINADB_COST_MODEL entry or join (the default since 2026-10-17), calibrated work admission (estimated serial work above four times the measured dispatch, morsel and merge overhead) replaces this floor for aggregates; row-producing pipes need both. In model-off mode (PATINADB_COST_MODEL=off), zero forces the work threshold but retains shape and specialization exclusions. The default is an unverified scheduling heuristic, not a measured crossover; results retain the serial contract. |
PATINADB_PAR_FANOUT_MIN_EDGES | 50000 | Minimum actual average first-hop key count per anchor before adjacency prefixes split at sampled keys (#805, #1442). Preparation counts pinned keys for small eligible anchor sets; below the threshold, execution uses anchor-only ranges. Edge ranges are capped at four times the query thread setting. Unverified performance heuristic; never affects results. |
--check-config validates PATINADB_QUERY_POOL_THREADS and
PATINADB_QUERY_MORSELS_PER_WORKER with the runtime’s exact unsigned integer
grammar and reports their clamped values. Zero becomes one; values above the
respective maxima become 32 or 64. Whitespace, suffixes, negative values and
integer overflow are rejected by the check instead of silently using defaults.
The gate declines to serial for anything it cannot prove safe: writes, AS OF
time travel, subqueries/CALL {}, variable-length traversal, pattern-property or
index-seek sources, secondary entry labels without a ready inverse index, sum/avg
aggregates, and DISTINCT projections in the chain. Historical #781 measurements
with the scoped scheduler (not the current pool): on the 8.4 GiB
1M-ticket benchmark database (warm page cache, 16-core box, min-of-3): a full
Ticket scan with count(*) + count(DISTINCT title) + min/max went
from 3 992 ms serial to 2 702 ms at 4 threads (1.48×) and 2 204 ms at 8
threads (1.81×); 2 threads is slightly slower (0.93×) than serial, so
those historical results do not establish a crossover for the current scheduler.
See RFC 0041 for the later coordinator measurements and their limits.
Covering reads
The #1399 covering path is selected automatically for eligible broad,
single-label integer-property aggregates. PATINADB_READ_COVERING=off is a
diagnostic off switch; on permits selection but never overrides eligibility.
Explicit parallel-query configuration retains existing parallel/serial admission
until its competition with covering is qualified. It keeps UUID encounter order, including
intermediate overflow behavior, using bounded external sorting when needed.
Float, mixed-type and wide-text columns, selective patterns, and unproved shapes
retain the existing path. Existing grouped-count fast paths keep precedence.
Limited synthetic qualification demonstrates the supported narrow-property
benefit under constrained total memory; it does not qualify every workload.
| Knob | Default | Purpose |
|---|---|---|
PATINADB_READ_COVERING | on | Automatic guarded selection; off retains existing access for diagnosis. |
PATINADB_READ_COVER_RUN_BYTES | 8388608 | Positive plain integer bytes for the run buffer, clamped to 64 KiB–64 MiB. Zero/invalid uses the default. |
PATINADB_READ_COVER_SPILL_BYTES | 1073741824 | Positive plain integer logical scratch limit including merge output. Zero/invalid uses the default. |
TMPDIR chooses scratch placement. Merge fan-in is limited to 16 readers and
one writer. Scratch cleanup occurs on normal completion, cancellation and errors.
The limits cover this operator’s auxiliary buffers and encoded scratch bytes;
filesystem allocation overhead, file cache, backend cache, aggregate group/DISTINCT
state and concurrent queries require their own memory/disk headroom. No total RSS
limit follows from a run budget. An estimated reorder larger than the scratch limit retains existing access
before preparation. Actual scratch use remains checked. If preparation exceeds its scratch budget
or cannot allocate/write scratch, it discards preparation and retains existing
access before emitting rows. Storage/index errors, cancellation and failures
after covering execution starts still propagate; ordinary aggregate limits remain.
EXPLAIN reports agg.covering_integer_uuid with row/byte estimates when the
candidate is selected, or a fallback reason. It separates estimated encoded key
bytes, reorder records, logical merge I/O and peak scratch. The secondary value
payload, page overhead and actual primary-read bytes are not included. These
estimates are not measured physical I/O costs or latency predictions. The 4096-row minimum is an admission heuristic, not a calibrated crossover.
PROFILE separates the planned access from observed aggregate execution,
including fallback reasons, actual covering record count and memory/external
reorder. Compare automatic selection with off under the same total RAM budget
when tuning a workload. See Query Planning.
Bulk import (sort-merge)
Offline bulk loads (patinadb import) use external sorting for both fresh
stores and re-imports, including --skip-ref-check. Mutation spools preserve
last-write-wins and removals. Existing stores maintain indexes incrementally;
fresh imports with duplicate IDs rebuild from the final primary state. Missing edge endpoints are still
skipped by storage. Ordinary online transaction/replication semantics are unchanged.
The old import opt-out and row-threshold selectors have been removed.
| Knob | Default | Purpose |
|---|---|---|
PATINADB_SORTMERGE_SCRATCH_DIR | $TMPDIR/patinadb-sortmerge-<pid>-<n> | Directory the external-sort spools spill to. |
PATINADB_SORTMERGE_RUN_BYTES | 128MiB | Per-spool in-memory run buffer before spilling to scratch. |
PATINADB_SORTMERGE_FILL_BATCH | 200000 | Entry-count cap for sorted-spool consumer batches. |
PATINADB_SORTMERGE_FILL_BYTES | 67108864 (64 MiB) | Key/value allocation-byte cap for sorted-spool consumer batches and direct snapshot-restore buffers (#1398). |
PATINADB_SORTMERGE_DELTA_WORK_BYTES | Automatic | Additional estimated-work budget per sorted incremental-delta transaction; does not affect fresh dense builds or snapshot filling. |
PATINADB_INDEX_BUILD_SORT_KEYS | 8000000 | Key-count cap for deferred index rebuild runs. |
PATINADB_INDEX_BUILD_SORT_BYTES | 67108864 (64 MiB) | Retained key allocation-byte cap for deferred index rebuild runs (#1398). |
The fill/rebuild byte knobs accept positive integer byte counts; zero or invalid values retain their defaults. Byte and count caps both apply. A single oversized entry is flushed alone. These controls are not total-memory guarantees: allocator, backend/cache and merge buffers need additional headroom. See deferred build memory.
The delta-work budget accepts positive integer bytes. Unset, zero or invalid values select automatic sizing: 1/32 of the configured redb cache, clamped to 1 byte–64 MiB. Without an explicit or pool-derived redb cache setting, the calculation uses redb 4.2’s 1-GiB fallback. Each delta is charged the larger of its retained key/value capacity (including descriptors and the temporary mutation tag) and 16 KiB. These are scheduling estimates, not measured page allocations. The existing fill-byte and entry-count limits still apply, whichever is reached first; an individually oversized entry makes progress alone. The cache setting is a sizing hint, not a reservation or a guarantee about an already-open database after a runtime cache-budget change. See delta tuning.
Fulltext construction also uses PATINADB_SORTMERGE_RUN_BYTES. Its scratch directories are private temporary
patina-fulltext-* directories under the process temporary directory; the
import-specific PATINADB_SORTMERGE_SCRATCH_DIR does not redirect them. Set
TMPDIR to a suitable disk before starting the process. The run buffer is not a
total RAM limit; merge readers, the largest posting list and database caches
need additional memory. Very small buffers create more runs and require more
open file descriptors during merging. See Full-Text Search.
Snapshot record encoding memory
Streaming historical snapshots, tag snapshots and anchor bases share an estimated 8-MiB input budget per encoding block, in addition to the existing 4096-record cap. Property values use the shared recursive byte estimator; borrowed records count too because their encoded output is materialized. One lookahead record may be retained, and a record larger than the budget is encoded alone without rejecting valid data. Record order, compression and wire framing are unchanged.
This is a scheduling estimate, not a total-memory limit: spare container capacity, the lookahead or oversized record, encoded output, codec contexts, allocator overhead and upstream database caches require additional memory. Multiple background writers have separate blocks. Shutdown is checked before and after reading each record, before encoding, before writing each record and before the end marker. A native read, compression call or filesystem operation already in progress cannot be interrupted by these checks. Publication and final durability still require successful completion; an aborted stream is not made visible as a complete snapshot.
Local checkpoint reuse during restart
During native startup, Patina may reuse its recovered local Raft checkpoint when the snapshot format, causal domain, applied log position and membership exactly match the current state. This avoids rebuilding that same checkpoint merely because peer versions have not yet been discovered. Checkpoints with mismatched state are not reused through this path; explicit repair retains its normal behavior. Sending snapshots to peers still requires the existing protocol compatibility checks.
A missing checkpoint, a moved data directory whose stored checkpoint path is no longer canonical, or a different state may still require a rebuild. The change does not establish a general restart-time guarantee or qualify the full large-database pressure soak (#1494).
Physical checkpoint copy pacing
PATINADB_CHECKPOINT_COPY_BYTES_PER_SECOND optionally paces physical Raft
checkpoint copying on each node. Set a positive integer number of input bytes
per second; unset or 0 retains the unpaced policy. Invalid values fail the
checkpoint build with a configuration error. The setting is read when a build
starts. A one-MiB burst is allowed; idle time does not accumulate extra credit.
This counts copied key/value bytes, not physical disk traffic. Native B-tree work, page-cache writeback, packaging, final syncs, historical tag reconstruction and other I/O are outside this limit. It is per build, not a shared-device cap. It does not change the snapshot cut, final durability or ordinary write policy. Cancellation remains checked while pacing; an in-flight filesystem call is not preempted.
Use this when simultaneous checkpoints compete with foreground commits on a shared device. A lower rate trades longer checkpoint creation and potentially longer log retention or learner onboarding for less copy pressure. Choose it from mixed-workload measurements on the actual storage and node count. This is an operator control, not a latency or memory guarantee; full low-memory cluster qualification is still pending.
Physical checkpoint package pacing
PATINADB_CHECKPOINT_PACKAGE_BYTES_PER_SECOND separately limits copying the
completed native checkpoint file into its hashed Raft snapshot package. Set a
positive integer byte rate; unset or 0 preserves the unpaced copy. Invalid
values fail the snapshot build. The setting is read for each file copy. It
counts actual file bytes, including any sparse regions represented in the
snapshot stream, with a one-MiB burst and a bounded 64-KiB read-ahead buffer.
Waiting checks shutdown; in-flight filesystem calls are not preempted.
This controls a different phase from native key/value checkpoint creation. It preserves framing, digest verification, file-length checks, final sync and atomic publication. It does not pace history serialization, native checkpoint construction, final syncs, restore, or ordinary requests. It is per file/node, not a shared-device cap or a foreground latency guarantee. Lower rates extend snapshot completion and log retention. Select rates with mixed-load measurements; the full low-memory soak is still unqualified.
A physical snapshot build releases its read views of the live databases before packaging starts. It first writes the history and restore streams to an unnamed temporary file in the snapshot directory, then writes every database’s checkpoint, and only then packages them. A slow package therefore no longer keeps old pages of the live stores pinned. The cost is temporary disk space: all checkpoints and the spooled streams exist at once until the build ends, where previously only one checkpoint existed at a time. The package bytes and their order are unchanged.
Historical tag-base fill pacing
PATINADB_TAG_BASE_FILL_BYTES_PER_SECOND controls the direct primary-tree fill
used when a background historical tag needs a reconstructed snapshot base.
A positive integer selects native key/value input bytes per second; unset or
0 keeps the existing unpaced policy. Invalid values fail the background build.
The four direct buffers share one budget per reconstruction and, when enabled,
flush at no more than one MiB of retained input each (a larger single entry is
still accepted alone). Existing smaller byte and entry caps still apply.
The rate counts data after decompression and native encoding, not the compressed snapshot file size. Waiting happens before a batch enters the native writer, and checks cancellation. Snapshot contents, publication ordering and terminal durability are unchanged. A lower rate delays tag readiness and can hold the history read lease longer; it is not a foreground latency guarantee.
This is a scoped background-work control, not a disk bandwidth or process-memory limit. The eligibility scan, edge-sort spools, metadata, delta replay, final snapshot serialization and fsync are outside this rate. Queryable AS OF restore, learner installation, live writes and legacy/genesis reconstruction fallback retain their normal policy. Use mixed-workload measurements for rate selection; low-memory full-cluster qualification remains pending.
Periodic snapshot scan pacing
PATINADB_SNAPSHOT_SCAN_BYTES_PER_SECOND controls decoded-record work for
periodic asynchronous history snapshots. The default is 16 MiB/s per process;
a positive integer selects another byte rate and 0 disables pacing. The node
reads the setting once on first use; restart to change it. Invalid values fail
the background snapshot build before publication.
Concurrent periodic builders share one budget and one MiB of burst credit, booked in arrival order so one builder cannot starve another. Each record is charged immediately after it is read, before the next iterator pull or encoding, using estimated decoded size with a 4-KiB minimum. Compressed output size would undercount wide properties. A pending record is charged only once. Its size is not known before reading, so one oversized record may exceed the burst; it remains valid. Budget waits poll shutdown every 50ms. An already running native read, codec or filesystem call is not preempted.
This prevents periodic snapshot workers from scanning freely just because their output compresses well. It is not a physical read/write bandwidth, dirty-page, RSS or latency bound: index traversal, cache misses, filesystem amplification and final fsync remain outside the estimate. Reads can flush dirty redb cache pages. Foreground exports/genesis/recovery, explicit tag exports, query scans, checkpoint copies and their existing budgets retain their policy. The record format, ordering, atomic rename and durable publication barriers are unchanged.
A lower rate delays snapshot readiness and keeps the read snapshot pinned longer, which can increase copy-on-write space and retained history. Size the budget for all nodes on the device and observe both foreground latency and disk growth. The default is an admission policy, not evidence of a passing full-size soak.
Private snapshot native-write pacing
PATINADB_SNAPSHOT_WRITE_BYTES_PER_SECOND optionally limits admission of native
file writes for private Raft checkpoints and historical tag-export bases and
overlays, and the copy of each completed checkpoint into its Raft snapshot
package. Set a positive integer byte rate; unset or 0 disables this additional
budget. Invalid values reject private-file creation. The process reads this
setting once on first use; restart the node to change it.
All these private files in one process share a single rate and one-MiB burst. Each native write is charged, including redb page rewrites and commit writes. Creating another file does not reset the budget. Admission waits occur before at-most-64-KiB native calls, observe shutdown and release the budget lock before I/O. Writers are booked in arrival order and wait without holding the budget lock, so a long checkpoint cannot starve a small tag export: each gets a share of the rate. A syscall already running is not preemptible. Cancellation fails the private artifact; it never reports an unwritten chunk as successful.
This complements the input limits above: B-tree and copy-on-write work may write
more bytes than the incoming records. It is not a physical-device bandwidth
limit: filesystem metadata/write amplification, reads, other .snap
serialization, live graph/Raft/audit writes and final sync waits are outside it.
The checkpoint package copy is charged per 64-KiB chunk before it is written,
after the optional per-file package rate above; with this budget unset
and no package rate, the copy stays unpaced.
Ordinary restore and query mirrors also retain their normal policy. Syncs, file
locking and publication order are unchanged.
A lower rate prolongs checkpoint/tag readiness, pins and log retention. Select it with the complete node count and mixed workload; multiplying the setting by the number of nodes still does not yield a device-I/O guarantee. The severe low-memory three-hour soak remains pending; this control is not a latency or memory guarantee.
Kill switches (rollback valves)
Presence-based env kill switches for recent read-path accelerations — set the
variable to any value (including =0) to disable the optimisation, e.g.
for A/B benchmarking or if you suspect a regression. Results are identical
either way; only the serving path changes.
PATINADB_DISABLE_QUERY_CTX_REUSE(issue #632) — disables the per-open schema-metadata cache and the per-DatasetcachedProcContext(both are rebuilt per query instead).PATINADB_DISABLE_EXTRA_LABELS_NEGCACHE(issue #631) — disables the negative cache on the extra-labels hydration probe (the per-row “does this vertex have secondary labels” check).
Time-travel performance knobs
These change only how an AS OF read is served, never what it returns — a
historical result is byte-identical whichever path runs. Both are read-path
accelerations with a fallback that is always correct, so they are safe to flip
either way; see Time Travel for the full
picture.
| Knob | Default | Purpose |
|---|---|---|
PATINADB_PARTIAL_TIMETRAVEL | on | Reconstructs only the subgraph an AS OF read actually touches, when the query’s labels/relationship types are statically bounded (an unbounded shape falls back to a full reconstruct). Read-only, so a write-only workload pays nothing. =0 is the rollback valve — set it if you ever suspect the partial path; the full reconstruct then serves every read, slower but identical. |
PATINADB_INDEXED_SNAPSHOTS | off | Builds a point-queryable .snapidx sidecar per snapshot so a cold single-vertex AS OF lookup is a point read instead of a full-graph rebuild. Opt-in because it costs disk. The build is lazy (triggered by a read that would benefit, never by a commit) and asynchronous (it never blocks the read that triggers it — that read is served by the scan path), so enabling it can’t stall a writer or the server’s apply loop. |
PATINADB_MAX_SNAPSHOT_INDEXES | 16 | Caps how many .snapidx sidecars may exist at once (0 = unlimited). Bounds sidecar count, not bytes — worst-case disk is roughly N × the size of a snapshot file. Only relevant when PATINADB_INDEXED_SNAPSHOTS is on. |
Why
PATINADB_INDEXED_SNAPSHOTSis opt-in. It earns its keep on repeated historical point reads: the read that triggers a build is served by the scan path, so the sidecar only helps later reads. If your backend time-travels rarely, or reads a different point each time, leave it off — you’d pay the disk without the win. Turn it on when a known set of historical points is read repeatedly. |--query-timeout-secs| server flag |300| Per-request wall-clock budget for a REST read; overrun →503.0= off. | |--max-concurrent-requests| server flag |512| Max in-flight client statements node-wide (REST and Bolt); excess shed with503/ a retryable Bolt error.0= unlimited. |
Set an env budget to 0 (or unset) to fall back to the built-in default. Tighten
them as DoS guards on a shared node, or raise PATINADB_MAX_HOPS for a
genuinely deep graph. A RETURN … LIMIT k is the normal way to bound result
size — there is no implicit result cap (an unlimited query streams every row).
For observability, set PATINADB_SLOW_QUERY_MS (server env, off by default)
to log a WARN for any REST query slower than that many milliseconds — carrying
the query’s normalized shape (literals + $params folded to ?), not raw
values. Per-shape latency stats are also served at GET /mgmt/queries.
Commercial entitlements
The server resolves a set of commercial caps (cluster HA size, combined
node+edge scale, database count, history-retention window, and two feature
gates) either from the hard-coded Community ceiling or from a signed license
token’s entitlement claims. This is configured entirely by which license you
install (--license / PATINADB_LICENSE / <db-root>/license.key — see
Licensing & Telemetry), not by a server flag. See
Editions & Limits for the full Community/Pro/Enterprise table,
what each limit does when you hit it, and how to read a running node’s
resolved tier + live usage via GET /version (also scraped onto
patinadb_entitlement_usage_ratio{axis} / patinadb_entitlement_limit{axis}
Prometheus gauges).
Transaction and query memory budgets
These budgets are active by default on the server: it installs one work-memory governor at startup from the automatic or configured values below. Embedded library use has no governor unless the caller installs one.
RFC 0036 server integration, bounded Q/TX/foreground controls and the original
120-second default smoke pass at 4bfb2d3e. The separate three-hour default-pressure
run passes the agreed runtime gate after independent and final review, with its
observed cap overshoots and sampling gaps. Performance measurements are complete;
performance acceptance remains open. The new write matrix is reviewed with an unresolved
COMMIT p99 increase; the completed 4bfb retained-tail comparison adds a higher
first-read latency. Main wires the four settings below
through ENV, CLI and YAML, with CLI > ENV > YAML > automatic defaults and the
same resolution used by --check-config. Source tests and read-only config checks
do not qualify a complete node under memory pressure.
The existing outer settings are listed in Cache memory budget.
With Main’s shared Limits.memory_governor, normal writes also check their
already resolved request before Raft proposal. Checked 6 * counted_data_bytes + 64
must fit the configured Apply allowance. The count covers retained container
layouts, accessible string/label capacities, nested values, old conditional
preimages, idempotency/CAS data and Anamnesis operations. Opaque text uses visible
UTF-8 bytes plus a header; this is a coarse allowance, not a query-text prediction
or an exact heap/RSS bound. Existing producer admission runs earlier and keeps its
owners; their accounted credit is not multiplied or reserved again. A separate
foreground allowance covers the check’s temporary nested-value descriptors.
A terminal refusal occurs before the rejected request/chunk is proposed, preserves
TX/Engram atomicity, and does not replay it. For chunked imports, successful earlier
chunks remain committed; the first terminal proposal error prevents later chunk
proposals even with ON ERROR CONTINUE. Ordinary resolvable query errors retain
their existing continuation behavior. Already committed Apply/replay is not
rejected by this gate. Qualification remains open; see the
normal-write scope.
Governed Bolt streaming reads
Bolt streaming reads now use their statement’s query allowance throughout the blocking Core worker, with native read fences for live and reconstructed plain historical stores. Existing covered parameter, expression, sort and materialized result paths share that scope. There is no new setting or per-row reset.
The context retains expression and binding credits until the worker exits, including dead intermediates. Long streams can therefore be refused even when their exact live heap would fit. Sort scratch keeps its own shorter lifetime. Encoded frames remain separately owned and release their credits as PULL/DISCARD or disconnect releases their storage. RESET/disconnect closes admission; worker data stays charged until it actually drops, without a synchronous-join guarantee.
Coverage is still partial: L3/L4 RowTable-hit outer copies and the ordinary lazy
stream’s hydrate_row property containers/recursive scalar hydration are not
admitted by merely installing this context. Native/cache/transport internals and
other uninstrumented producers also remain separate. Focused controls and broad
regression validation pass; performance qualification for this change remains open.
See the streaming scope and controls.
The completed bounded comparison at 08bd4971 still exceeds the first-read
overhead objective. A later change prepares existing cache-policy metadata
during enabled governor construction. Its bounded query/retained comparison now
completes with improved first-read medians, but adverse individual blocks remain;
a general 10% bound and final-source pressure qualification remain open. See the
measured results and source bindings.
Earlier integration scope
The following producer notes record the separately qualified source increments. Their statements that an individual increment added no switch/default describe that earlier scope; the server now supplies the shared governor by default. They remain coverage boundaries, not a full-memory or performance guarantee.
Core reservation and shared Raft
operation ownership are implemented. An explicit Rust Limits.memory_governor
can additionally admit Bolt explicit-TX operation capture/retention and encoded
RECORD buffers through PULL/DISCARD. Encoding borrows the Core table and reserves
its own iterative scratch, output, framing overlap and shared byte owner before
allocation. Partial delivery retains those credits; terminal delivery destroys
its wire storage before SUCCESS. Assembled Bolt message storage is also reserved
before reading each chunk; decoded PackStream layouts retain their permits with
the parsed request. Explicit RUN input and result work use the same statement
allowance. An oversized incomplete frame aborts unproposed work and closes the
connection; a complete frame rejected during decoding allows RESET afterwards.
Explicit-TX RUN conversion additionally reserves parameter map storage, keys,
nested containers/strings, expanded byte lists, point coordinates and named-zone
storage. Worker clones share one immutable owner and its credit. Conversion
refusal aborts unproposed TX work and allows RESET. The admitted private Core
resolve paths also reserve runtime parameter maps and fresh nested values before
binding. Their returned owner keeps parameter backing charged through worker and
result aliases, until RECORD encoding has produced independent bytes. The central
RowTable hydrator additionally reserves its concrete output containers, property
decoding, labels and fresh scalar values as it builds them; the result owner
retains their credit through delivery. Oversized output can therefore refuse RUN
even when the mutation payload fits. This preserves the private resolve boundary
and does not create a partial Engram. Other output producers still need coverage;
this introduces no new switch or automatic default activation.
Explicit admitted Bolt COMMIT also reserves its new CAS token containers,
fresh values, label arrays and sorting scratch; local request/entry clones share
those data owners. Old vertex/edge properties for the final CAS tokens are now
decoded directly into admitted storage. Their strings, containers and iterative
decoder scratch are reserved before allocation; read/decoder errors propagate
instead of becoming missing properties. Refusal occurs before proposal and
reports no effect. Constraint catalog construction, native
page/cache/registry storage, encoded overlay read buffers, the interned label
arena, fallback preimages and other write paths remain outside coverage.
A large old property can therefore make COMMIT refuse even when the small update
itself fitted RUN. The wire schema and server defaults are unchanged; full
qualification is still pending.
The common column-oriented QueryResult hydrator also uses the active development
query scope. Before constructing its output maps, row arrays and scratch, it
reserves a coarse 4096-byte allowance per actual input cell. This is a counting
policy over rows already produced by execution, not a prediction from Cypher.
Repeated columns and freed scratch can be overcharged. The allowance is credit;
it does not cause an allocation of that size. Stored properties and copied value
payloads are admitted separately, and output credit remains with the result owner.
The explicit development governor also connects ordinary REST data reads to this
same execution scope. Covered working buffers can reject a query even when its
final response would be small. Core callers opt in via query_read_with_memory;
this data-read API rejects writes and fences native mutation of the live graph
on the executing thread. Historical plain stores are fenced too; private lazy
overlays keep their reconstruction behavior. Other threads may still write.
The fence preserves atomicity on refusal; it is not a general procedure sandbox.
Result-cache hits reserve the same coarse container policy plus fresh payload copies before creating their QueryResult. Credit remains with the result owner after query sealing or cache eviction. This deliberately copies payload backing; its performance impact has not yet been qualified. Persistent cache admission, L4 decode buffers, earlier row/value producers and alternate emitters remain separate gaps. This introduces no new setting or production default activation.
With the explicit development governor, both expression evaluators now share admission for argument arrays, list/map expression containers, comprehension output and generated integer lists. The existing buffer budget checks actual requested capacity before allocation and old/new overlap before growth. A typed resource failure stops the admitted producer and reaches the normal query error boundary; no write is proposed by a failed private resolve. Argument slots release after the call. Returned container credit stays with the query/result owner, conservatively including dead intermediates until that owner drops. This can reject work sooner than exact live-heap accounting. Strings, recursive value clones, entity/row payloads and remaining producer paths need separate admission; these container credits do not establish complete coverage or an RSS bound. Existing row caps, semantics and error types remain. No new setting/default is activated.
The development governor now also applies one coarse allowance before scalar
string production: 6 * work_bytes + 64, with checked arithmetic. This is a
counting policy, not six times measured RSS. trim variants, string reverse,
left, and each split piece use actual UTF-8 input bytes. ASCII case conversion
uses the same byte count; non-ASCII lower/upper conversion uses three times input
bytes to allow expansion. replace counts actual non-overlapping matches (also
empty patterns) and uses the larger of input and output bytes. substring and
right use four times input bytes to include their existing char-array scratch.
A 500-byte ordinary string therefore reserves 3064 credits, without allocating
3064 bytes merely for accounting. Split also admits its growing list slots.
The shared arithmetic executor also applies this allowance to + string
concatenation, using the combined UTF-8 byte lengths of the actual values. A
mixed i64/f64/bool operand adds 512 work bytes for the existing scalar formatter
before the sixfold allowance. That fixed allowance is a policy, not its measured
RSS. The same path serves both expression evaluators. List concatenation admits
its growing output slots and moves existing elements into them, preserving nested
payload storage instead of recursively cloning it. Existing source buffers still
need their own admission and remain separate from the new output slots. The
query/result owner retains output credit. Consumed string operands release their
own credits only after new output construction finishes; operands whose data may
be moved into results retain their owners. Other producers can still retain dead
intermediates conservatively. No parsing-based cost prediction, default or ENV switch is added.
See concatenation validation.
Covered literal, parameter and scalar-binding copies in the expression executors now share runtime admission. Reading an existing string shares its EcoString backing (or copies its inline bytes) without a new expression payload charge (#1449). Source storage remains a separate obligation; newly computed strings still require producer credit. Recursive values reuse the checked container/payload copier, including map keys. Materialized table results keep fresh admitted payload copies so their delivery owner is independent of source rows. Inline strings allocate no heap payload, so the copier checks scope admission with zero payload credit; their bytes are covered by value/container storage. Heap copies reserve their fresh payload layout. String functions retain their existing producer allowance. The scalar dispatcher borrows its arguments and admits selected/copied output instead of eagerly cloning its first argument. Consumed list/map indexing moves the selected payload. Output credits remain with the query/result owner, conservatively including dead values. This is scoped coverage; other row/AST/value copies and producers remain open. See runtime value-copy validation.
The augmented runtime rows used by both expression evaluators for local bindings now reserve their actual container capacity and copied outer payloads before allocation. Mutating shared row backing first admits a fresh copy through the same helper. Lexical shadowing and evaluation order remain unchanged. Source storage and inserted values still require producer ownership; copied row credit stays with the enclosing query/result owner, conservatively including dead values. Other row producers remain uncovered. This adds no keyword-based prediction or configuration switch. See runtime binding-row validation.
The library transformations and Unicode behavior remain intact. Credit is reserved before the call and retained with the query/result owner; dead temporary strings can stay charged until that owner drops. These estimates may reject work before its exact live heap reaches the limit. Existing value ownership, other conversions, remaining recursive clones, native memory and remaining producer paths are separate. All-scalar Unicode and real allocation-request probes support the policy for the tested toolchain and cases; allocator slack/RSS are not proven by those probes. Performance impact is unmeasured. No new knob/default is enabled.
The Bolt metadata correction adds patinadb_effect: "none" only for
Neo.ClientError.Statement.ResourceExhausted when discarding an open unproposed
explicit TX with no pending committed result, or in read-stream setup/execution.
Other errors keep their existing metadata. This describes persistent mutation;
read records may already have been delivered. Ingress/decode refusal while a
committed write/control result is still pending does not gain this annotation. It does not convert a lost commit
acknowledgement into a safe replay. The original bounded TX control exposed missing
effect metadata and remains FAIL. A new bounded control at 4bfb2d3e completes
strict query/TX/foreground refusal, no-effect, subsequent progress and five-node
owner-release checks. The full run completes eight in-mix Q/TX controls and a
separate post-mix foreground control with the same strict oracles; independent
and final review accept this finite workload. Limits and retry policy are unchanged. See the
boundary validation ledger.
The same development governor now covers runtime operation capture and pinned
CAS/OCC for ordinary REST /cypher autocommit writes in both existing modes.
Execution and old-value token growth share one query budget across pipeline
attempts. Pre-proposal resource refusal reports HTTP 400 with
Neo.ClientError.Statement.ResourceExhausted, retryable: false and
patinadb_effect: "none". Outstanding worker/proposal owners keep their credits
after the HTTP future drops. Ordinary REST data reads and writes also reserve
converted parameter containers/keys/values and the worker query copy before
allocation. Handler and workers share this immutable input; writes charge it
to the same execution/CAS allowance. The /cypher HTTP collector reserves wire
buffer growth from that same query scope before allocation, including streamed
requests without Content-Length. Axum’s existing body-size limit (413) and JSON
validation remain independent; exhausted memory gives the resource response
above. Before the unchanged JSON decoder runs, an additional conservative
allowance is reserved from that same scope and retained until the decoded request
is destroyed. Successful decode releases only the separate wire buffer.
The JSON allowance is a byte-based accounting policy, not a Cypher cost prediction:
wire_bytes * (6 + 3 * floor(log2(escape_count + 1))) + structure_bytes.
structure_bytes starts at 512 bytes, adds 1024 per opening object brace or
backslash, 128 per opening array bracket, and 256 per comma. Counts include quoted
strings: the existing decoder can parse embedded JSON strings through its
raw_value support. Escapes allow multiple live decoder buffers; Unicode-escaped
markers are included conservatively. All arithmetic is checked. These are policy
coefficients, not measured RSS ratios. Quoted punctuation, duplicate keys and
ignored fields can be overcharged; the complete allowance remains reserved during
conversion/execution even after parser scratch has been freed. Insufficient
allowance produces the resource response before JSON parsing or mutation, so it
can precede a syntax error. There is no new tuning switch for these coefficients.
Ordinary REST data-read responses also retain a budgeted byte buffer through
HTTP body ownership and transport clones. The unchanged JSON serializer writes
through the existing admitted vector: capacity growth is checked before allocation,
including simultaneous old/new buffers. This charges actual buffer capacity;
it does not allocate six copies of the estimated output. A growth refusal discards
the partial buffer before sending HTTP 200 and returns HTTP 400 / resource
exhaustion with retryable: false. This is an output-stage error, not the
pre-dispatch patinadb_effect: "none" input contract. Completed writes and control
responses keep their existing response path. No new switch or default is added.
Native transport frames, allocator overhead, rendered error bodies, downstream provenance copies, uncovered read operators/result emitters, JSON-tree construction, and other REST routes remain uncovered. Budgeted encoded bytes do not make the whole result pipeline bounded. The server supplies the shared governor from the ENV/CLI/YAML settings below; complete qualification remains open.
With the development governor, OCC now uses admitted hash sets for entity, property and replaced-label identities. This covers the candidate, each intervening Engram’s structural marks, and pinned CAS construction in both Bolt write modes. A pipelined candidate keeps its table credits while its in-flight registry entry or a conflict-check worker still owns it. Exhaustion is a pre-proposal resource error with no effect, not a conflict retry or optional-CAS fallback. A small write can therefore refuse while checking a large unrelated commit since BEGIN; retrying a new transaction after RESET starts from a fresh base. Retained constraint maps, label arrays, copied values, prefilter sets and additive UNIQUE tuples now use that allowance too. They keep their credits across history pages; replacing a map value releases its old payload only after admitting the new one. This keeps the existing final-value and additive-UNIQUE semantics. Emitted keys and their scratch tables/values are now admitted too; their owners keep credit after the accumulator is dropped, including while waiting in the pipeline. Composite-key variants are emitted individually with the existing expansion cap and numeric equality rules. Governed store-property reads for these checks now decode under the same allowance, retaining the source credit through key construction. Missing properties remain distinct from memory/read errors. Buffered final values are borrowed; explicit deletes do not read the live value again. Live and pinned read views retain their original boundaries. Decoded label IDs and label-result arrays in OCC and pinned CAS capture now reserve before allocation too. Missing vertices are distinct from read or memory failures. CAS retains the same admitted array after removing its primary label; membership/order semantics are unchanged. The retained list of intervening Engram IDs now admits capacity growth before allocation too, including temporary old/new-buffer overlap. Metadata strings are borrowed during the walk. Many small foreign commits can therefore exhaust the allowance even if each individual body is small; this returns a resource error before proposal, not a conflict retry. Governed OCC now also decodes each stored body directly into admitted operation/value storage. RAW bytes and metadata strings are borrowed; decompressed ZSTD output remains charged while decoding. Each body owner drops after its fold; the shared constraint context survives across bodies. A refusal remains a resource error before proposal. The native ZSTD context/window, tree/overlay/cache buffers, interned label arena and catalog source data still need separate admission. No additional toggle or automatic default is activated. These development controls are not a full memory-pressure qualification or a measurement of the PR’s performance impact. With the development governor, streaming Bolt output also reserves its encoded and framed bytes before allocation. Queued frames and transport writes retain their credits until sent/discarded; an idle consumer can exhaust the byte budget before the 256-row channel fills. A mid-stream refusal emits FAILURE after any valid prefix, without a successful completion/bookmark, and aborts an admitted explicit transaction’s unproposed writes. This reuses the existing RECORD encoder; it does not cover the Core executor’s earlier copies/rows or fixed channel storage. Control messages currently share foreground capacity too; a full pool can close the connection instead of admitting RESET. This development API does not meter handshake/dispatch, provenance, streaming executor parameter copies, per-evaluation value copies and result tables/operators, result metadata, mirrors, TLS/WebSocket internal buffers, native caches or committed replay. The server passes one governor to all explicit integration points; that wiring does not make this a qualified memory profile. Complete coverage and pressure qualification are still required before release.
Settings and automatic defaults
The defaults use effective RAM: physical RAM constrained by the process/container’s applicable cgroup limits. They do not grow with database file size. A large file can be read through bounded buffers; a small database can still receive a transaction with very large values. The own-heap planning budget leaves room for page cache, runtime and allocator usage; it is not a hard RSS or kernel-memory cap. Caching being off must not disable transaction protection.
Query text and database size cannot establish a reliable allocation bound. The governor admits covered actual buffer/container growth during execution; optimizer estimates may guide a plan but do not replace that admission. Existing values must be decoded under their allowance too, not decoded fully and checked afterwards. This is dynamic accounting, not a predicted maximum query payload.
Reservations include buffer capacity and temporary overlapping buffers, not just the final value or wire size. Bolt RECORD encoding starts with an admitted 256-byte output buffer and space for the row’s work descriptors, then admits further growth as needed. A deliberately tiny query budget can therefore reject even a small result. This uses the existing query/transaction/pool limits; there is no separate encoder setting.
Names and automatic formulas (integer bytes, rounded down):
| ENV | CLI flag / YAML key | Meaning | auto value (default) |
|---|---|---|---|
PATINADB_MAX_TX_MEMORY_BYTES | --max-tx-memory-bytes / max_tx_memory_bytes | Live accounted bytes across all statements of one client transaction | min(256 MiB, foreground / 4) |
PATINADB_MAX_QUERY_MEMORY_BYTES | --max-query-memory-bytes / max_query_memory_bytes | Statement sublimit within its transaction, shared by parallel operators | min(128 MiB, transaction / 2) |
PATINADB_APPLY_MEM_LIMIT | --apply-mem-limit / apply_mem_limit | Protected committed-apply, replay and recovery capacity | 25% of work |
PATINADB_BACKGROUND_MEM_LIMIT | --background-mem-limit / background_mem_limit | Shared scratch capacity for optional maintenance | 10% of work |
The outer defaults keep min_free = max(1 GiB, TOTAL / 10) and
MEMORY_LIMIT = TOTAL - min_free. Cache allowance, work and runtime headroom
use 40%, 45% and 15% of MEMORY_LIMIT; foreground is work minus apply and
background. TX/query caps sit within foreground, not on top of it. These are
allowances rather than per-connection preallocations. The resolver rejects zero
usable capacity and inconsistent partitions; successful configuration resolution
alone is not proof that every supported entry fits the Apply/replay envelope.
Unavailable discovery requires an explicit absolute own-heap ceiling and must
remain reported as unknown, not invented host RAM.
On the server, PATINADB_CACHE_LIMIT=0 disables entry
caches, but neither disables the work-memory governor nor sets native cache
capacity to zero. An explicitly empty or whitespace-only value (for example,
PATINADB_CACHE_LIMIT='') also keeps entry caches disabled, as before. The native
cache allowance in these cases retains the automatic share of 40% of
PATINADB_MEMORY_LIMIT.
Admission discovery follows the process’s actual controller membership and mount information. V2 checks visible ancestor limits; v1 uses the kernel’s hierarchical memory limit. Unreadable or ambiguous controller data prevents automatic sizing; a manual ceiling must still fit any bound that was observed. A subtree-only v2 mount requires an explicit budget because ancestors above it are unavailable. Namespace-hidden ancestors and changes during/after discovery remain limitations; this startup snapshot does not prove exclusive available RAM. The independent entry-cache free-floor sampler still manages eviction; it does not choose another startup allowance or override a refused admission budget.
The server connects its shared work governor to the existing Linux process-metrics tick (every two seconds), using the original startup discovery and budget. It rechecks visible memory limits. A known bound below the original startup bound stops new positive foreground and optional-background reservations, even if an explicitly smaller heap still fits. When startup had no known bound, the comparison uses the checked sum of own-heap and minimum-free allowances. Missing data does not mean zero; a bound observed despite other missing data still counts. Limits and the original comparison baseline are never raised by this monitor. Raising the cgroup limit again does not reopen admission: restore a valid configuration and restart. Existing allocation owners remain charged, and protected committed Apply/recovery is not cancelled or skipped.
The node logs the reduction and sets the unlabeled Prometheus gauge
patinadb_memory_admission_stopped to 1 (initially 0). New positive client
reservations receive the existing typed PATINADB_WORK_MEM_LIMIT resource
error. This sampled monitor is not a hard response deadline, cannot immediately
shrink existing objects and cannot guarantee protection against an OOM between
samples. It runs on Linux; there is no cgroup monitor on other platforms.
The four new controls accept positive byte counts, binary units such as MiB
and GiB, or auto. Bare numbers are bytes. The checked parser treats KB/MB/
GB aliases as binary too; prefer explicit KiB/MiB/GiB. Fractional bytes
round down. Zero, negative values, overflow, malformed input and percentage
syntax in these four settings are errors. There is no unlimited switch for
these new budgets; existing settings retain their documented zero behavior.
Precedence is explicit CLI flag, then ENV, then YAML, then automatic default.
Startup and --check-config share one checked resolution, retaining discovery
and any problems. The dry run prints resolved budget bytes, discovery and the
read-only inventoried native plan; it installs no governor, reserves no leases
and opens no native database. A valid plan does not prove complete path coverage.
For a 2 GiB effective ceiling, automatic arithmetic produces 1 GiB own-heap,
314,069,483 foreground bytes, 78,517,370 per transaction and 39,258,685 per query.
The bounded control and seven default-smoke generations at 4bfb2d3e observe
these settings without memory overrides (PR #1410; see the
startup validation ledger).
The older proposed 64 MiB / 32 MiB profile below is only an explicit-override
example, not the current managed-default qualification profile.
An 8 GiB ceiling reaches the automatic
256 MiB / 128 MiB TX/query ceilings; more RAM does not keep increasing those
per-request caps. A ceiling of 1 GiB or less leaves no own-heap with the
1 GiB minimum free allowance and is refused; smaller-node support would need an
explicit, separately qualified partition. The bounded 2 GiB observations and
other arithmetic examples do not establish generally safe defaults. The
protected apply budget must handle the supported committed-entry envelope
before any default can ship, including after restart and on every replica.
Equivalent explicit overrides for the historical proposed example:
PATINADB_MAX_TX_MEMORY_BYTES=64MiB
PATINADB_MAX_QUERY_MEMORY_BYTES=32MiB
PATINADB_APPLY_MEM_LIMIT=auto
PATINADB_BACKGROUND_MEM_LIMIT=auto
max_tx_memory_bytes: 64MiB
max_query_memory_bytes: 32MiB
apply_mem_limit: auto
background_mem_limit: auto
Matching flags are --max-tx-memory-bytes 64MiB --max-query-memory-bytes 32MiB --apply-mem-limit auto --background-mem-limit auto. Current configuration/parser, precedence,
automatic-default and read-only --check-config controls pass as recorded in the
startup validation ledger.
These source checks are separate from full pressure qualification.
Rejecting a client transaction must preserve
atomicity; a timeout after proposal still requires the existing outcome/readback
rules. A replica cannot reject an already committed entry based on its local
client budget. See limitations.
Cache memory budget
For an overview of how these regions nest, with calculated values for 2 GiB and 16 GiB, see How Storage Works → Memory model.
patinaDB can keep a governed, RAM-budgeted cache of decoded objects / adjacency
/ query results above the OS page cache. The server defaults it ON, at
PATINADB_CACHE_LIMIT=40% of the resolved memory limit, when you leave the
flag/env var/config key unset — so a fresh node ships with caching already
engaged. Set PATINADB_CACHE_LIMIT=0 (or the equivalent flag/config key)
explicitly to disable the four decoded entry caches. On the server,
native stores keep their inventoried aggregate allowance and work admission
stays active; this is not an unrestricted-memory or zero-runtime-cost mode.
The server prepares the existing empty 4 MiB W-TinyLFU frequency
sketch while constructing an enabled governor, before query admission. This
moves initialization cost to construction; it records no synthetic access,
warms no query result and adds no ENV/CLI/YAML setting or changed default.
Disabled construction stays lazy. An embedded caller that later enables its
disabled governor with set_budget retains the existing lazy behavior. Source
checks pass. Its bounded performance comparison is recorded in the linked ledger;
it does not establish a general latency bound or new-source pressure qualification.
The budget uses the shared cgroup-aware discovery described above: observed
host RAM and all accessible controller bounds constrain the effective total.
Missing or ambiguous hierarchy data cannot become a successful host-RAM fallback.
This observed ceiling is not a guarantee against OOM or other consumers.
From that total, PATINADB_MEMORY_LIMIT is patinaDB’s own-heap ceiling (not
including the OS page cache), and four regions are carved from it. Each knob is
an absolute size (8GiB), a fraction of its parent (40%), or auto.
Value precedence is CLI > ENV > YAML > default, regardless of the value’s
format. Fractions are evaluated against the resolved parent.
| Env var | Region | Default | Parent |
|---|---|---|---|
PATINADB_MEMORY_LIMIT | own-heap ceiling (excl. page cache) | auto = TOTAL − min_free | discovered TOTAL |
PATINADB_CACHE_LIMIT | Combined entry/native cache allowance; 0 disables entries only, retaining the automatic native allowance | 40% | MEMORY_LIMIT |
PATINADB_WORK_MEM_LIMIT | action reserve (concurrent query working memory) | 45% | MEMORY_LIMIT |
PATINADB_MEM_HEADROOM | transient-spike / allocator-slop / OOM safety | 15% | MEMORY_LIMIT |
PATINADB_CACHE_MIN_FREE | page-cache floor (system free RAM kept resident for the storage engine’s page cache) | max(1GiB, 10%) | discovered TOTAL |
Every knob is also a patinadb-raft flag (--memory-limit, --cache-limit,
--work-mem-limit, --mem-headroom, --cache-min-free) and a YAML config key
(memory_limit, cache_limit, work_mem_limit, mem_headroom,
cache_min_free), with the same explicit-flag/env > file > default
precedence as every other setting.
The budget is validated at startup and fails loud (the node refuses to boot)
when it over-commits — CACHE_LIMIT + WORK_MEM_LIMIT + HEADROOM ≤ MEMORY_LIMIT
and MEMORY_LIMIT + CACHE_MIN_FREE ≤ TOTAL — with an error naming the offending
setting or partition. Successful startup logs the resolved budget in bytes;
--check-config prints those values when memory resolution succeeds.
These budgets are not a hard process-RSS limit. The enabled sampler caps host
MemAvailable by each readable cgroup-v2 ancestor’s remaining allowance plus
clean, inactive file cache. It uses each ancestor’s own usage, including sibling
processes. Dirty/writeback and shared-memory pages are not credited as immediately
reclaimable. Missing cgroup data retains the host fallback; missing all data does
not invent memory pressure. This is a conservative, non-atomic estimate, not an
allocation-admission guarantee. The existing cache free-floor feedback now uses
this estimate, so abundant host RAM cannot hide pressure inside a container.
On Linux/glibc, the enabled sampler also requests release of already-freed
allocator pages every 30 seconds (#1306). Below CACHE_MIN_FREE it may request
this sooner, at most once per second and no faster than the sampler cadence.
After a reclamation attempt the sampler reads availability again before the
cache free-floor policy decides whether useful entries must be evicted.
Those are maintenance policy intervals, not measured latency or RSS guarantees.
Reclamation does not discard live data, invalidate indexes or change configured
cache limits. Fragmented pages containing live allocations can remain resident.
Other targets do not use this glibc-specific maintenance. Keep headroom and
monitor actual process/cgroup memory. A smaller cache or periodic reclamation
cannot guarantee that a concurrent workload will fit its hard memory limit.
Example — a container with memory.max=16GiB, PATINADB_CACHE_LIMIT=40% and
otherwise defaults: TOTAL=16GiB, min_free≈1.6GiB, MEMORY_LIMIT≈14.4GiB
(TOTAL − min_free), then Cache ≈5.8GiB · Actions ≈6.5GiB · Headroom ≈2.2GiB.
Set PATINADB_MEMORY_LIMIT=10GiB to leave more RAM to the page cache on a
read-heavy deployment.
The core decoded caches share 4 * floor(cache_limit / 8) bytes. Normally
objects, properties, adjacency and L3 results each receive one quarter of that
allowance. When PATINADB_L3_MAX_RESULT_BYTES is below the fixed 256-byte L3
entry overhead (for example 1), L3 cannot admit entries and the other three
core levels split its unused share equally. This environment setting is resolved
once; restart after changing it. Zero or invalid values retain the 1 MiB default.
L4 is disk-backed and has no decoded share to donate: setting
PATINADB_L4_VICTIM_MAX_BYTES=0 still disables only that victim tier.
With otherwise default budgets, the calculated adjacency cap at a 2 GiB cgroup
limit becomes 68.267 MiB with L3 disabled, versus 51.2 MiB with L3 enabled
(#1442, static budget arithmetic). This is not a measured speedup or a guarantee
that a working set fits. Native-cache requests, the free-memory floor, work
budgets and auxiliary-cache caps are unchanged. The sampler can still evict under
pressure. An 8 GiB cgroup with PATINADB_MEMORY_LIMIT=1GiB and
PATINADB_CACHE_MIN_FREE=1GiB reproduces that application budget for diagnosis;
it does not reproduce kernel reclaim under a 2 GiB hard limit.
These env-only tuning knobs sit next to the budget and are validated
by --check-config:
| Env var | Default | Effect |
|---|---|---|
PATINADB_REDB_CACHE_LIMIT | Inventoried aggregate sizing below | Exact per-primary-file native cache request. Unset selects automatic sizing; 0 or empty requests redb’s built-in 1 GiB, never unlimited/disabled. Binary byte sizes and percentages of the same discovered TOTAL are accepted. Invalid, overflowing or non-Unicode values, percentages without reliable discovery and impossible aggregate requests refuse startup and --check-config; startup requests are never clamped. auto is not an override value: unset it instead. |
PATINADB_PRIMARY_POINT_IO | on | Linux primary-property point reads use a separate no-read-ahead handle when allocated database size exceeds detected RAM at open. off retains native I/O for diagnosis. Standard boolean values apply; reopening is required. Ranges and writes retain the native handle. |
PATINADB_CACHE_ADJACENCY_MAX_EDGES | 8192 | Per-list admission ceiling for the L2 adjacency cache, in edges (issue #808). A fully-drained incident-edge list longer than this is never cached — a supernode-scale single-touch scan keeps streaming off the raw storage range scan instead of flooding the cache and displacing small hot anchors. 0 disables the ceiling (not recommended: giant scans then churn the level). Performance-only; never affects results. |
Disposable redb stores (historical reconstruction, resolve scratch, physical
Raft checkpoint candidates and the L4 victim tier) use at most 64 MiB of native page cache per file, or the smaller
configured live-file cache. With the exact live-file override set
to 0, scratch requests remain capped at 64 MiB; disabling entry caches still
uses the aggregate automatic native sizing described below. They do not silently inherit
redb’s 1-GiB default. This is a per-store cache limit: concurrent stores, in-memory
scratch backing, staging buffers, query results and the OS page cache still need
headroom within a process/cgroup limit.
The RFC 0036 development API cache::native::install(CacheCapacity { total, entries, protected }) additionally reserves all native opens against total - entries.
The four standard entry levels share entries; a share that rounds to zero
keeps entry admission disabled, including after budget changes. Install
before cache registration and with no live native handles. The split is immutable;
it does not resize existing caches. protected keeps part of total - entries
unavailable to temporary opens; it must fit that native remainder. Persistent
graph/Raft/auxiliary stores can use the protected floor and otherwise free native
capacity, always subject to the total. Ordinary temporary/scratch/checkpoint/
compaction opens share only the unprotected remainder and refuse if their requested
cache does not fit. The explicit required-open distinction below handles the
listed committed/recovery paths without changing ordinary optional admission.
Native capacity remains charged through owned
read guards, open retries and failed-open cleanup. The existing per-file override
(including its zero/default request) cannot bypass this allowance. This API is
selected by the server through the shared
StartupMemory::prepare function, without another ENV/CLI/YAML switch. Startup
plans known graph/Raft/audit owners before opening files and
holds one required Fork/Squash reconstruction slot: it uses the existing
scratch size, retains the same charge through final native-owner release, and
adds no ENV setting. Native/Sim bounded ownership/progress controls pass;
this is not complete runtime qualification. The explicit sizing API below covers
known startup owners and Main now selects it; complete Apply/recovery memory
qualification remains open.
Configured capacity is not RSS: native/pinned/teardown overhead and in-memory
backing remain outside this cache counter.
The server calls StartupStorePlan::prepare_sized through
StartupMemory::prepare. It derives fixed per-file requests from the same
resolved aggregate cache allowance C and one physical startup inventory. The
matching entry-cache governor is installed before native policy and before any
entry cache/native open. The plan consumes that inventory once and transfers the
original reservations to registry, Raft, audit and required reconstruction owners;
opening a file does not reserve twice. --check-config resolves the same budget,
strict REDB override and inventoried formula read-only; it installs no policy,
reserves no slots and opens no native database. Neither API is a promise that
all future databases or overlapping recovery operations fit.
The explicit plan keeps E = 4 * floor(C / 8) for the four entry caches, or E = 0
when they are disabled. D includes every actual graph file plus missing
default/system; Raft trees share one file and audit has one separate file.
With N = C - E and s = min(p, 64 MiB), the revised automatic plan chooses the
largest positive whole-byte p such that (D + 1) * p + (2D + 8) * s <= N.
Required startup owners reserve (D + 1) * p + 2 * s, including audit and the
held required-reconstruction owner. The remaining role allowance accounts for
D retained tag-export bases, D Raft snapshot mirrors, one sequential checkpoint,
two scratch owners for each of the two shared periodic/tag build slots, and
one additional foreground scratch owner. Periodic and tag work share those two
slots; they are not separate concurrency allowances. Existing builds can overlap
a Raft snapshot because its inhibitor prevents new starts without draining them.
This divides the unchanged aggregate native allowance into smaller automatic
per-file requests. It neither reserves these optional roles against other queries
nor guarantees capacity for arbitrary foreground concurrency, historical-query
caches, newly created databases or retained old generations. One foreground
scratch owner is not a whole-transaction concurrency guarantee. Native active
pages, in-memory backing and other allocations remain outside this configured
cache capacity; it is not an RSS limit. The revised role calculation passes source
and real-owner integration checks after a second pre-mix smoke refusal. The
subsequent bounded automatic-default control and original smoke at 4bfb2d3e
pass, followed by the agreed full pressure qualification after final review.
This does not cover arbitrary native-owner concurrency or establish a universal
capacity bound. Performance acceptance remains open; the completed write and
retained-tail comparisons preserve their separate latency findings.
The explicit API takes an already-parsed exact REDB override: None is automatic,
Some(bytes) is never clamped, and Some(0) still requests redb’s native 1 GiB
cache. The exact primary request and its bounded scratch requests must fit the
complete role plan above or startup and --check-config reject; no override is
silently reduced. When entry caches are enabled, a matching entry governor must be installed first; live native
owners or registered entry caches reject later policy installation. The fixed policy is retained after
later startup I/O failure while leases follow actual owner lifetime. Startup
must terminate after such a failure; retrying a different configuration in the
same process cannot replace the once-only policy. The agreed default-enabled
pressure qualification is complete; arbitrary required-memory capacity and
performance acceptance remain outside that result. See the
sizing contract and source validation.
The required-native increment, with focused source validation, gives explicitly required post-startup persistent opens the smaller of their request and currently available native cache allowance, including zero. It covers committed Create/Anamnesis/Fork targets, received snapshot probes/replacements and legacy record restore. Pending or forced install repair uses the smaller of request, native remainder and temporary remainder for its mirrors/checkpoint before clearing the durable intent. Ordinary opens and periodic snapshots retain strict admission; required temporary work cannot consume the protected persistent floor. Leases track granted bytes and retain their real native-owner lifetime, including old/new overlap and zero byte owners. Reduced grants warn; real storage errors still fail.
Databases created after startup (CREATE DATABASE, fork targets, Anamnesis
companions, record-restore creates) receive native cache only beyond the startup
plan’s temporary inventory — the capacity kept for snapshot mirrors, checkpoints,
export bases and the foreground mirror (#1499). Automatic sizing gives all other
capacity to the files inventoried at startup, so with an unset
PATINADB_REDB_CACHE_LIMIT a newly created database runs with a zero-byte
configured cache until the next restart re-plans across all graph files; capacity
returned by a dropped database is reused. An explicit override smaller than the
automatic size leaves the difference for later databases. Earlier builds applied
a protocol-17 CREATE DATABASE with strict admission: where the free share was
smaller than one planned request (every large host under automatic sizing) the
committed entry was a fatal apply error and the node aborted, also on each replay.
The startup-planned server path is covered by an accounting test, not an RSS or
performance measurement; the cost of zero configured cache is unmeasured.
An internal granted zero is not PATINADB_REDB_CACHE_LIMIT=0: that ENV value
still requests 1 GiB, and strict startup must fit the exact request. Zero configured
cache does not eliminate active/pinned/write pages, native metadata or in-memory
mirror backing. The required-open path itself adds no switch or durability/deadline
change; server activation is described above. See the
required-open and repair scope.
The separate development Limits::memory_governor integration also admits
/raft/append wire buffers from the shared protected Apply pool. It preserves
PATINADB_RAFT_MAX_BODY_BYTES, including on chunked bodies. A retryable peer 503
with code=PeerAppendMemory means the wire buffer or decoded operation payload
was refused before Raft handoff; it does not promise the entry had no effect
elsewhere. Decoded operation arrays and recursive values retain their Apply
credits through the final OperationBatch owner. The server uses this same governor, without a separate transport
memory toggle. Full default activation qualification remains pending. The outer entry array, metadata/CAS fields,
interned identifiers, parser/native transport scratch, vote ingress and complete
committed apply/replay still need coverage and capacity qualification. Stored
entries are not yet proven to fit every configured Apply pool; transport retries
alone do not provide that progress guarantee.
Retained-tail boot recovery streams current-format entries after full preflight without a new setting. It preserves group durability and keeps a private temporary hash file beside the retained tail; scratch creation/write errors prevent restore. There is no supported memory cap for individual stored entries from this change. Live forced installs also save a pinned native log range one entry at a time, then reuse that replay path, without an additional switch. Missing/corrupt source logs still require reseeding; retained-copy write errors precede graph restore. Legacy v1 boot recovery uses an anonymous copied tail and the same verifier. A bounded runtime Apply diagnostic confirms another open protected-path gap: OpenRaft’s replication entry cap does not bound full committed-range decoding or multiple decoded batches queued behind a held consumer. After release all checked values apply correctly; this is a memory-scheduling gap, not observed data loss. The diagnosis alone did not qualify a scheduler fix or default activation. See Apply queue diagnosis.
The bounded Apply integration now selects the reviewed OpenRaft source. With the explicit development governor, the serial worker loads at most 64 whole entries and respects the existing encoded-byte reader target; an oversized first stored entry still progresses intact. Pending ranges coalesce without crossing snapshot commands, and source logs survive until durable Apply/install permits purge. These are scheduling bounds, not an individual entry’s decoded-memory or native-page bound. Focused integration and Raft/Sim/crash controls pass. The subsequent Bolt boundary correction resolves the earlier fixture failures with fixed test allowances and additional small-cap refusal controls. Complete resource qualification, defaults and the new pressure soak remain open. No ENV toggle is added by this integration.
Raft log append now serializes entries one at a time inside the same atomic native write transaction. Each encoded buffer is released before generating the next; success is acknowledged only after the complete durable commit. Generation or storage failure cannot publish a partial generated batch. This removes the whole-batch encoded staging buffer; input entries, individual encoded values, native transaction pages and complete protected Apply ownership remain separate obligations. Those log-stream prerequisites alone enable no default. Runtime cost still needs measurement. See atomic append validation.
With the explicit development governor, ordinary committed boot replay also streams individual entries before Raft construction. Its original 64-entry batch boundaries and durable barriers stay intact. Each batch is decoded once for preflight and again for apply, so startup cost still needs measurement. Corrupt or missing committed records prevent readiness. The fault-injection build keeps its existing asynchronous recovery seam. Non-Apply complete log readers, per-entry expansion and complete protected/native capacity still need qualification; the streaming-replay prerequisite alone does not qualify default activation.
Default asynchronous snapshot commits attempt retention pruning without waiting for historical readers. A busy lease leaves all snapshots intact and retries on the next snapshot-taking commit; the retention count remains a soft cap. Explicit synchronous snapshot mode retains its bounded wait. Read consistency and snapshot publication/durability rules do not change.
The primary-point policy uses the host/cgroup RAM ceiling, including the process’s nested cgroup-v2 scope and tighter visible ancestor limits. It is a coarse eligibility check, not a query working-set or RSS bound. New empty stores and stores that fit that ceiling keep native I/O until reopen re-evaluates them. EXPLAIN shows the policy; PROFILE reports query-thread backend requests, not physical disk traffic. Unavailable Linux file advice falls back to native I/O.
Internal defaults (informational)
These are mostly not user-configurable flags today, but are useful to know:
| Setting | Value |
|---|---|
| Engram (history) snapshot interval, per database | every 50 commits, or earlier once 1 MiB of deltas accumulated (PATINADB_SNAPSHOT_DELTA_BYTES; 0 = count only) |
| Raft snapshot interval | every 5,000 log entries (PATINADB_SNAPSHOT_LOGS_SINCE_LAST; 0 or invalid = default) |
| Raft election timeout | 750–1500 ms |
| Raft heartbeat | 250 ms |
| BM25 parameters | k1 = 1.2, b = 0.75 |
| Full-text prefix/fuzzy expansion cap | 256 terms |
| Bolt streaming channel | 256 records (bounded) |
| Default database name | default |
On-disk layout
The directory and table map is in
How Storage Works → Where your data lives.
The --db directory contains the B-tree tables for the graph, the
property/compound indexes, the engram log and snapshots, the full-text catalog
and index data, and the persistent Raft log and state-machine metadata. Back
up the whole directory as a unit. Each database is a subdirectory of --db.
Entry and join cost model
PATINADB_COST_MODEL accepts exactly off, entry or join (case-sensitive), once per
process. Since 2026-10-17 unset or empty means join (#1442). off restores the
reference planner; any other value also selects off, as before the default change, and
--check-config rejects it. entry compares whole-path costs for labelled entries in fixed-length
paths without a sorting hint. UUID, sorting, shortest and variable-length policies
remain unchanged. EXPLAIN/PROFILE record entry.cost alternatives in estimated
nanoseconds, output rows, provenance, the dominant term and assumptions.
entry.cost.hops describes estimated scanned and accepted rows for each hop.
entry.cost.consumer adds work a consumer repeats per run of one binding; today
only the count over independent OPTIONAL one-hop branches is priced.
These values are experimental estimates, not latency predictions (#1442).
The constants use A2-cal primitive measurements; full query estimates still assume
independent filters. Warm redb is the default; warm L2 is estimated only for lists
within the admission limit and a working set within the enabled level cap. Cold
is never selected automatically. Secondary checks use inline costs only with
A1 observed compact-payload coverage. MANY and NO_INLINE observations retain
membership cost, and the current view must still be READY. These are cost assumptions, not cache-hit guarantees.
join includes entry and additionally estimates fixed-pattern join outputs from
population-conditioned degrees. Hash build/probe and seeded Apply traversal
compete in estimated nanoseconds. Small join chains use subset DP, medium chains
use interval DP over a greedy ordering, and larger chains use greedy search.
The result replaces the reference join plan only when the same model predicts an
advantage of more than 1.1x (all inputs exact) or 2x (sampled, default or stale
inputs). A join chain directly under count(*) whose reference plan is rewritten
into grouped per-side counts keeps that plan if the result would lose the
rewrite; EXPLAIN reports kept reference: it executes as a partial COUNT.
Chains without any relationship, chains with a cycle-closing pattern
(a node variable repeated inside one pattern), unsupported paths and refused
planning scratch retain the reference join planner. The margins are unverified
policy choices, not measured optima (#1442).
join.order and join.strategy show the selected plan’s alternatives and
provenance. Cached join plans retain their original decisions and schema epoch;
data/statistics refreshes do not replan them. PROFILE marks unavailable operator
counts explicitly. Q-errors describe estimates, not correctness checks.
With entry or join, the model also replaces the PATINADB_PAR_AGG_MIN_ROWS floor for
parallel admission (parallel.admission reports serial and parallel estimates)
and enables the auto values of PATINADB_WCOJ and PATINADB_CSR. Without
a cost model, auto behaves as off. These switches change only plan and operator
choice, never results (#1442).
PATINADB_DEGREE_STATS_EDGE_BUDGET is an unsigned edge budget, read once per
process (default 1000000; 0 disables background refresh). With entry or join
(the default), missing or stale A1 groups request a single background sample, so the
model works without a manual CALL patinadb.stats.degrees(). A group missing from an
unsampled catalog requests nothing while its edge type and both labels still have
current rows, because that catalog scanned every edge and the pattern direction has
none (#1442). The requesting query uses
cached legacy fanout or a default immediately. It never waits for this refresh.
The worker has a cooperative 120-second deadline and observes global shutdown;
its construction memory and published catalog use the governed stats.degree
region. The worker reserves what its group ceiling needs (about 13 MiB for the
4,096-group maximum), or what is left of the region if that is less, but at
least enough for 256 groups. The granted bytes then cap the groups and the
encoded catalog, so small nodes (2 GiB) still get a sample. A catalog that would
exceed the grant is not published (#1442). Refusal preserves the fallback. Population counts still visit vertices;
the edge budget alone is not a vertex-work bound. There is no new background work
when the model is off. Samples assume uniform UUIDs. A conservative confidence
margin retains the reference entry for weak legacy/stale advantages or sampled
estimates whose apparent advantage is below their cardinality disagreement.
What the background sample means in practice:
- The first queries of a shape after start, after a database is created and after writes that make a group stale plan with the fallback (reference or legacy-fanout estimates). Plans can change once the sample is published; results never do.
- The sample is node-local RAM. It is not persisted or replicated and is rebuilt after a restart. Snapshot and overlay reads never start one. One worker runs per process, with retries for the same database spaced at least 5 seconds apart.
- The work is not free: up to the edge budget of sampled relationships plus one pass
over all vertices for the population counts, on one background thread. A disabled
entry cache (embedded use without
PATINADB_CACHE_LIMIT), or one too small for the 16 MiB construction reservation, refuses it; the model then works on default estimates with its widest (2×) adoption margin. - Join orders (
join) are part of the cached plan: one planned before the sample arrived stays until a schema change or eviction. Entry choices are re-priced when the catalog changes. CALL patinadb.stats.degrees()still refreshes exactly (or with an explicit edge budget) on demand, for example as a warm-up after bulk loads.
Restoring the reference behaviour
The four RFC 0042 switches are on by default since 2026-10-17 (#1442): the measured
candidate gave Velr 0.544, LSQB 0.878 and SF1 0.995 of all-off (geometric means of
median ratios, three server rounds; RFC 0042 A6). Set all four to off to restore the
previous planner and executor exactly:
PATINADB_COST_MODEL=off PATINADB_FACTORIZE=off PATINADB_WCOJ=off PATINADB_CSR=off
Each switch can also be turned off alone. PATINADB_COST_MODEL=off alone leaves
PATINADB_FACTORIZE on (every proven shape then engages, uncosted) and makes the
auto modes of the other two inert. Every switch is read once per process; restart
after a change. Explicitly set values keep the meaning they had before the default
change; only unset or empty values changed. --check-config prints the effective
mode of each switch, marking defaults. PATINADB_QUERY_THREADS stays 1.
Pipelined write snapshot admission
Before executing a pipelined Cypher write, Bolt and REST may wait for a complete native read snapshot within the existing query deadline. Only snapshot acquisition is repeated: this does not replay the query, a WASM function, or a submitted mutation. Other storage errors propagate. If the query timeout is disabled, or the calling thread already owns a deferred native write, an unavailable publication is refused rather than waited on. The network server’s existing default query timeout remains 300 seconds; no new setting is added. The focused publication-race regression is verified; renewed full pressure qualification of this change is still pending.