Introduction

This manual documents patinaDB 0.22.0.

Revision 0.22.0

Hardens backup/recovery, ordered mutation execution, query resource limits and transaction lifetimes. Adds mixed-direction partial-prefix ordering and release SBOM/notice verification. See 0.22.0 Release Notes for compatibility boundaries and the pending candidate qualification. Historical soak results do not qualify this new candidate; publication is still separate.

Revision 0.21.0

Improves eligible cold filtered traversals and aggregates, preserves procedure bindings after WITH and in correlated subqueries, and adds independently checked WASM workload coverage. See 0.21.0 Release Notes for the separate six-hour non-WASM and thirty-minute WASM qualification results. Release preparation does not imply that tagged artifacts are already published.

Revision 0.20.0

Adds stable multi-key ordered-edge pagination with residual filters and ordinary graph continuation. This revision also includes the maintenance jobs, recovery, diagnostic tracing and cache improvements merged since 0.17.0. See 0.20.0 Release Notes for the three-hour mixed qualification, upgrade boundaries and remaining cold-aggregation and author-page latency gaps. The version number is not a claim of universal production readiness.

Revision 0.17.0

Fulltext index construction groups postings through a bounded external-sort buffer instead of repeatedly rewriting each term list for every document. Storage format and synchronous incremental maintenance remain unchanged. See 0.17.0 Release Notes for measurements and boundaries.

Revision 0.16.0

Adds bounded parallel vector-index training, cached initialization distances, and incremental candidate consumption during vector search. See 0.16.0 Release Notes for scope and validation.

Revision 0.15.1

Portable restore parses its bounded upload incrementally instead of buffering the full JSON document first. Administrators can explicitly configure a larger restore byte cap; decoded graph/history objects still require RAM. See Backup & Disaster Recovery. The revision also fixes the backup qualification harness; no production approval follows from the bump.

Revision 0.15.0

This revision adds region-aware Bolt routing and scoped causal bookmarks, includes the post-0.13.0 correctness/security fixes, and provides automated release qualification. Read the 0.15.0 release notes for upgrade boundaries and the still-open concurrent-onboarding latency gate. Publication of this manual is not production approval or a binary release.

Revision 0.13.0

This revision preserves valid property-population metadata when existing property values are overwritten, including replicated bulk writes. It is a generic storage correction, not a Jira-specific query rewrite (#1261). It also adds actual snapshot/purge indices to the operator metrics and reproducible logged-fixture worker restart tests. See First Customer for measured local recovery coverage and the required backend retry behavior. A version bump is not WAN qualification or production release approval; see Limitations.

patinaDB is a managed graph database server, deployed as a Docker container. It stores a property graph — vertices (nodes) and directed edges (relationships), each carrying typed properties — on top of an embedded, single-file ACID B-tree storage engine with native cross-tree atomic commits. You query it with a Cypher-like language over your choice of two wire protocols, and every write is recorded as an engram so you can inspect history and travel back in time.

patinaDB (patinadb-raft) is a standalone node that replicates writes with the Raft consensus protocol, hosts multiple databases, and speaks both a JSON REST API and the native Bolt protocol used by Neo4j drivers and the Neo4j Browser. A single node with --bootstrap is a drop-in lightweight server (a “cluster” of one, no consensus-latency penalty for a single voter); add peers to get automatic failover.

What makes patinaDB distinct

• Versioned by construction. Every mutation is committed as an engram — a git-like commit for your graph. List the history, view a git-style diff of any single change, diff two arbitrary points in time, tag a meaningful state, and run read queries against the past with USE … AS OF. No audit table to bolt on — the history is the database.
• Provenance, for free. Because the history is a first-class change stream, patinaDB’s Anamnesis system auto-projects it into a queryable PROV graph in a companion database — who / what / when, and optional source / confidence / derived_from at commit, type, attribute, or per-entity granularity, enriched straight from Neo4j transaction metadata. Where Neo4j leaves you to assemble this from APOC triggers or CDC, here it ships in the engine. See Anamnesis.
• Cypher-compatible. The engine passes ~97% of the openCypher Technology Compatibility Kit (TCK). See Cypher Support for exactly what is and isn’t covered.
• Neo4j-tooling-compatible. The server’s Bolt endpoint works with the official Neo4j drivers and the Neo4j Browser, so you can point existing tools straight at patinaDB.
• Search is built in. Full-text search with BM25 ranking and vector/embedding search (an IVF ANN index, cosine & euclidean) ship in the engine — with the same procedure syntax as Neo4j (db.index.fulltext.queryNodes, db.index.vector.queryNodes). No external search service, no separate vector database.
• Scales down and up. A one-node cluster (quorum of 1) behaves like a plain lightweight server with no consensus latency; add peers for automatic failover (High Availability). Snapshots and time-travel reconstruction stream in O(chunk) memory, so the graph isn’t bounded by RAM.

Who this manual is for

This is the user manual — how to install, query, operate, and integrate patinaDB. It documents every user-facing feature and, just as importantly, its limitations.

A note on scope

patinaDB deliberately implements a subset of Neo4j/Cypher. It is not a drop-in Neo4j replacement for every workload — it targets versioned/auditable graphs and small-to-medium replicated deployments. Where behaviour differs from Neo4j, this manual calls it out explicitly. Read the Limitations chapter before committing to a production workload.

0.22.0 Release Notes

Released 2026-10-25 from the merge of PR #1510 (4254bca0, tag v0.22.0). It includes everything since 0.21.0. The full list is in the repository changelog.

Highlights

• History epochs (RFC-0038). Opt-in sealing, archiving and cluster-wide hot release of old history; released history is read from the verified archive. See History epochs.
• Identity and access management (RFC0037). Profiles, API keys, optional OIDC/OAuth providers and SCIM-managed identities. See IAM.
• Database incarnations. Each database carries a replicated incarnation UUID; drop and recreate, and restore, start a new one.
• Query engine. The RFC 0042 planner and executor switches are on by default (factorization, degree statistics, cost-based entry choice), with bounded per-query memory admission across sort, join, UNWIND and collect paths. See Query planning.
• Storage. Secondary labels moved inline into the adjacency (schema 8, RFC 0040 increment 1); wide vertex-property values are stored once per owner (schema 7). See Storage.
• Stability under memory and disk pressure (PR #1510). Paced checkpoint packaging, batched and recoverable squash cleanup, bounded redb freed-page reclamation per commit, single-barrier fork seeding, and a BEGIN wait bounded by the transaction deadline.

Data safety and recovery

Bulk upserts enforce schema constraints, keep graph entities intact across chunks and preserve unambiguous pre-load rollback targets. Restored baselines are checkpointed before admitting replicas, with snapshot publication and purge ordering enforced. Backup archive v2 retains property registrations and named indexes that older exports could lose.

Forced snapshot replacement durably retains the replay tail or fences the incomplete repair. Backfill errors are classified per call and retain their original cause; repair latches remain database-local across restart. Vector preparation propagates scan errors and publishes complete definitions last.

Queries and operational limits

Completed deferred tag snapshots release their cached read transactions, including when another snapshot path completed the work. Durable tags and pending snapshot captures remain intact. This removes a source of retained old database pages; fresh long-run memory qualification remains required.

Spill-heavy index loading now merges at most 32 input runs at a time, adding one output file for an intermediate merge. This fixes reproduced file-descriptor exhaustion while preserving duplicate ordering and cleanup on failure or early drop. The bound applies to each active merge, not all descriptors in the process; no throughput improvement is claimed. See the implementation and regression evidence.

FOREACH executes ordered mutation clauses with lexical scope and authorization. Actual nodes/relationships retain their identity through containers and Bolt, JSON and WASM conversions; UUID scalars remain scalars and real Paths stay Paths. Store scalar references such as id(n) rather than entity/path property values.

Grouped ordering preserves renamed keys. Eligible partial-prefix multi-key ordering supports mixed directions while retaining NULL/tie-break rules, materialization limits and execution errors through SKIP. Seeded UNWIND and FORK DATABASE share their corresponding row/database limits. Terminating an idle transaction wakes its owner. Cache shape-admission history is bounded. Decoded rotation request secrets are zeroized on drop, including cancellation; this is not a guarantee about every raw transport or parser buffer.

COUNT subqueries in projections and WHERE comparisons share a seeded plan, including correlated filters. Single-pattern COUNT accumulates matches without materializing result rows; evaluation errors remain errors. General list-predicate lowering and decorrelation are still pending. Eligible unsorted equality filters use bounded advisory index probes, and SHOW INDEXES includes compound definitions.

Authorized management audit responses include stored peer_ip and origin. Both embedded UIs replace the vulnerable single-file plugin dependency chain with a shared offline bundler. These changes do not inherit the earlier soak.

Upgrade and downgrade boundaries

• Storage schema is 8. Opening a schema-7 store migrates it in place and rebuilds the inline label data; schema-6 stores must be recreated or reimported (no on-disk migration). Do not open a migrated store with an older binary.
• Peer protocol is 22 and the Raft snapshot format is 8. Physical snapshots with schema-8 bodies require protocol 22. Feature floors (incarnations 17, IAM 19, epoch metadata 20, history restore 21) apply once the corresponding records exist; new members must then speak at least that version. A matching protocol number alone does not qualify every mixed-version scenario.
• New backup readers accept archive formats 1 and 2. Old readers reject format 2; verify restores with compatible binaries before retiring older exports.
• Local snapshot-install intent changes from 1 to 2 and retains v1 reading. Do not downgrade across an unfinished v2 install/recovery intent.
• Direct Rust runtime-value consumers must handle the added Entity variant. JSON containers expose structured entities; Bolt returns native entity containers. Entity/path property values are rejected rather than coerced; store id(n) explicitly.

Qualification

PR #1510’s three-hour five-node mixed pressure soak passed on the released product code (run ae8ab7dd43ed, independently audited, no OOM), and its CI passed all 47 checks. This is one passing soak on one host, not a latency guarantee: the write tail under squash and checkpoint load is reduced but not solved (#1516, #1512). The RFC-0024 token release gate (#1151) remains a separate decision, including acceptance of finite token retention and eviction at capacity. A release is not a production approval for any particular deployment.

0.21.0 Release Notes

This revision includes the cold-query improvements merged after 0.20.0 and the seeded-procedure correction. Release preparation, published artifacts and production approval remain separate steps; check the repository release status.

Query execution

• Eligible filtered aggregates reuse exact indexed string values rather than repeatedly reading graph properties. Unsupported shapes retain the ordinary executor, including null, mixed-domain and resource-budget fallbacks.
• Long ordered-edge traversals can prepare selective residual candidates; short pages continue streaming. Filters, ordering and multiplicity remain part of query semantics, not application-specific rewrites.
• Parallel workers preserve filtered entry selection, pinned parameters and bounded aggregate covering. Query parallelism remains opt-in.
• Bounded top-K sorting enforces the retained-row budget, including offset windows, and reports resource exhaustion instead of truncating answers.

Procedure correctness and WASM

Procedure calls after WITH and inside correlated subqueries now retain imported row bindings. Previously the fallback could invoke a procedure with unbound arguments; a real WASM guest reproduced the failure. The fix is generic and preserves database-scoped procedure resolution.

The WASM manual explains bounded application calls, backend authorization before invocation, module registration and trust boundaries. WASM remains subject to its existing experimental-feature requirements.

Qualification and limits — #1261

The preceding frozen cold-query candidate passed twelve verified cold starts with 24 correct answers and a strict six-hour mixed run with 113,327 successful timed HTTP attempts. This was a non-WASM workload, not a six-hour WASM test. Exact artifacts and prior failures are recorded in repository docs/cold-query-qualification-1261.md.

The subsequent procedure-fix candidate passed fresh five-node WASM controls, including learner restart, module replication and a deliberately unavailable module. Only the failing WASM client stopped; ordinary reads continued. A separate thirty-minute mixed run passed all 11,393 timed HTTP attempts, including 360 independently checked WASM page requests. WASM HTTP p90/p99 were 250.5/465.2 ms, including backend and transport, not isolated engine time. See repository docs/wasm-mixed-1261.md for hashes, rates and raw evidence.

Both workloads used one SSD host with three voters, two nonvoting learners, three Rust/neo4rs backends and simulated regional delays. The fixture contains 334,770 tickets, 2,201,041 comments and 5,488,094 edges, not one million tickets. The measurements predate the metadata-only version bump; they are not new measurements of a published release binary.

This does not qualify real WAN deployments, maximum write throughput, batch synchronization, arbitrary hostile guests or every cold query shape. No storage or peer-protocol format changes are introduced by this revision.

0.20.0 Release Notes

This revision consolidates the changes since 0.17.0. There are no implied 0.18.0 or 0.19.0 releases. Release preparation, tagged artifacts and production qualification are separate steps; consult the repository release/tag status before deploying an artifact described by this manual.

Ordered graph pages

Edge-sorted indexes now support eligible multi-key pages, necessary primary cursor bounds, deterministic residual filters and ordinary additional graph hops. The executor orders complete primary-value families before selecting secondary keys. It preserves matching-row multiplicity and checks filters before counting page rows. Existing index definitions and persisted entries are reused; this feature does not require rebuilding them.

Planner and executor controls cover the unchanged backend-shaped query, identity projections, snapshots, ties, cursor boundaries, errors and cancellation. Unproved query shapes and primary-value domains retain the general executor. Selective filters, large equality groups and offset pagination may still require substantial work. Use keyset pagination where appropriate; no fixed candidate count or universal latency guarantee is implied.

Other changes since 0.17.0

• Maintenance jobs: asynchronous tag and backup work, observable node-local execution and history maintenance operations.
• Backup and recovery: retained snapshot-body ownership, streaming/portable restore and offline administrative recovery controls.
• Property-aware result-cache dependencies, snapshot-safe adjacency reuse and L4 cache visibility in the admin dashboard.
• Diagnostic tracing for logical query, search and snapshot phases, plus regional workload, causal-read and recovery qualification tooling.
• Recovery, latest-onboarding I/O and routing hardening. These changes do not remove the documented application retry, licensing or experimental-feature requirements.

Measured qualification and limits — #1261

The frozen pagination candidate before the metadata-only version bump completed a strict 10,800-second mixed workload on 2026-09-21. It used one SSD host, three voters, two nonvoting learners, three Rust/neo4rs backends and simulated 20/40 ms one-way learner delays. The dataset contained 334,770 tickets, 2,201,041 comments and 5,488,094 source edges. It was not a 1M-ticket test.

All 56,620 timed HTTP attempts succeeded and the independent result/order, cursor, ACL, causal-read and search checks passed. The final run retained host caches and included concurrent mutations and analytics; it did not inject faults.

These are nearest-lower sample quantiles, including regional delay; preflight and cleanup are excluded. The earlier initial smoke failed on first-touch author aggregation at the eight-second request budget. A subsequent green mixed run does not fix or waive that cold-path failure. Author-page tails also remain a performance follow-up. The isolated improved cursor case is not representative of every author page.

Reproducible evidence, exact binary hashes and failed attempts are recorded in repository docs/ordered-edge-pagination-1261.md and issue #1261. No real-WAN, cold-start, memory-limited or universal production approval follows from this version number or the three-hour result.

Upgrade boundary

All workspace crates inherit version 0.20.0. The ordered-edge feature introduces no new persistent index format. Intermediate changes since 0.17.0 include disposable cache-format changes; cached results are not authoritative database contents. Follow the first-customer checklist, preserve a verified recoverable backup and validate your own queries and driver behavior before rollout. Do not assume arbitrary mixed-version or downgrade safety from the unchanged edge-index format alone.

0.17.0 Release Notes

This documents the 0.17.0 release candidate. Publication of this manual is not a release tag, a merged-PR assertion or first-customer production approval.

0.17.0 optimizes initial fulltext index construction and reconstruction (#1261). It retains the existing posting-list storage format, analyzers, BM25 scoring and synchronous maintenance of subsequent document updates.

Grouped fulltext construction

Instead of loading and rewriting a growing posting list for every document, the builder stages document postings, sorts them and writes each complete term list once. Document encounter order is preserved within each list. Node and relationship indexes use the same builder; graph scan errors propagate rather than silently producing an incomplete index.

Sorting reuses PATINADB_SORTMERGE_RUN_BYTES (default 128 MiB) and temporary files under the process temporary directory (TMPDIR on Unix). This bounds the sort run buffer, not total process memory: merge readers, the largest posting list, serialization and database caches need additional RAM. Provide temporary disk space; ordinary completion, cancellation and errors remove scratch files, but a hard process kill can leave temporary directories behind.

Index creation remains synchronous and uses the existing atomic DDL boundary. No thread-pool setting, asynchronous update mode or on-disk migration is added. Existing indexes need not be rebuilt for compatibility.

Validation and scope

On 334,770 real Apache Jira summaries, standard analyzer, one summary property, local Ryzen 7 3800X/NVMe, the paired 2026-09-10 runs under #1261 measured:

The measured build is 217.31 times faster; this is one complete build per implementation, not a hardware-independent guarantee. Warm Top-10/20 p90 across the tested forms is 2.173–63.868 ms. No search algorithm change is claimed.

Differential tests cover all four analyzers, in-memory and spilled construction, posting bytes, query scores, subsequent updates and deletion. A regression guard checks one final write per distinct term list. The Jira benchmark checks exact hit sets, independent single-term BM25, Top-10/20, update visibility and reopen. Measured results are recorded in docs/fulltext-benchmark-1261.md under #1261.

This is not a qualification of 10M tickets, large ticket bodies, WAN fan-out, high-rate writes or concurrent queries during initial construction. Subsequent updates still rewrite affected lists. See Full-Text Search.

0.16.0 Release Notes

0.16.0 optimizes the existing IVF-Flat vector implementation (#1283). It does not introduce Lucene, HNSW, a new persisted index format or quantization.

Vector-index construction

Initialization caches each training sample’s nearest selected-centroid score. Each newly selected centroid updates that cache; previously selected centroids are not rescored during every selection round. Parallel work computes per-sample scores and Lloyd assignments. Weighted selection and centroid accumulation keep their original order, preserving existing seeded results.

PATINADB_VECTOR_BUILD_THREADS controls the shared per-process training pool: positive integers are capped at 8; 1 selects serial execution. When unset or invalid, the default is available logical CPUs minus one, bounded to 1–8. The setting is read at first training use; restart to change it. Failure to create the pool logs a warning and falls back to serial execution. Backfill persistence remains ordered and single-writer; this setting does not parallelize all of Raft.

Training logs report initialization completion and each Lloyd iteration. Search feeds decoded candidates directly into top-k selection rather than first retaining all decoded candidate vectors. This removes one intermediate buffer; it does not promise constant total RAM, a fixed index cache budget or bounded materialization in every historical storage view.

Compatibility and validation

CALL db.awaitIndexes([timeoutSeconds]) now checks local index readiness, without output columns. Synchronous index maintenance normally needs no wait; pending deferred builds obey the timeout and query cancellation. It is not a replication barrier. See Procedures. The compatibility reference also explains deliberate static-label/type syntax, client-side grouping, unsupported index hints/scoped CALL, and supported scalar vector similarity.

Measured on the local Ryzen 7 3800X / NVMe fixture with 334,770 real Jira summary vectors (#1283, 2026-09-09): the complete 256D index build drops from 3429.950 s to 72.759 s (47.14 times faster), using eight training workers and unchanged index parameters. Peak build memory stays about 1.18 GiB. Warm loopback Bolt p90 is 22.886 ms for Top-10 and 23.402 ms for Top-20; sampled recall against the same exact oracle remains 96.719% / 94.844%. These are single-client measurements on this fixture, not a 10M-vector guarantee. The source-tree ledger docs/vector-benchmark-1283.md records hashes, commands, 512D results, RAM and the intentionally interrupted legacy 512D baseline.

• Existing index definitions and similarity scores are unchanged.
• Reference tests compare centroid bits for cosine/euclidean, different worker counts and seeds, duplicates, empty/identical inputs and cluster counts.
• An operation-count test guards against reintroducing repeated initialization scoring. The optional timing benchmark compares the original implementation with the optimized one; wall-clock thresholds are not a flaky CI assertion.
• vector_training benchmarks a retained real database and checks its trained definition against the existing reference without replacing the index.
• The new openCypher Compatibility Reference documents supported, partial and unsupported forms. Its TCK inventory is explicitly dated to the reviewed 0.15.1 core revision, not a new 0.16.0 language-conformance certification.

These changes do not close the existing large-scale backup/restore, cold learner onboarding or 10M-vector qualification gaps. A version bump is not a production release approval. See Limitations.

patinaDB 0.15.0

All workspace crates share version 0.15.0. This manual covers the current revision; publishing documentation does not publish a binary or authorize a production rollout.

What changed since 0.13.0

• Region-aware Bolt routing includes named nonvoting learners and explicit local-read policies. See Multi-Region Topology.
• Scoped causal bookmarks fence reads against applied commits. A routed reader must catch up or refuse the read; routing alone does not guarantee freshness. See Bolt and Client Retry Contract.
• The correctness/security audit hardens snapshot database reconciliation, transport trust, Bolt resource limits and derived-cache lifetime. See Limitations for boundaries that still apply.
• Automated local release gates exercise idempotency, recovery, routing and impaired learner links. The Jira fan-out harness adds simultaneous admission under reads and updates. A local regression pass is not customer acceptance.

Current qualification: not production approved

The measurements below used the original relative gate. Subsequently, on 2026-09-08 (#1261), the user selected p90 <= 300 ms for explicit backup/fan-out profiles, with relative 3x increases reported as warnings. Old artifacts retain their original verdicts; this policy change is not a fresh qualification run.

The corrected-proxy local 1M-ticket run on 2026-09-08 (#1261) completed 60/60 successful samples per query family, with 2 reads/s and 0.1 updates/s during the mixed phase. Both learner logs show physical installation and subsequent history backfill completion. However, the unchanged whole-phase p90 gate fails:

The required limit is 3x each query’s idle p90. Q2/Q10 pass; only the previously accepted Q6/Q8 exceptions apply at this scale. Low absolute latency does not erase a failed relative budget. These are two 300-second phases on one shared host with synthetic learner links, not real WAN, isolated transfer latency, p99 qualification or a 6M/60M extrapolation. The run failed before final causal evidence was persisted; it does not establish a complete onboarding verdict. Source: #1261, docs/jira-fanout-2026-09-08.md in the source tree and its retained million2 raw baseline/mixed reports. The interference cause remains unisolated.

Cold onboarding at the target throughput, real-backend integration, real WAN, off-host restore and operating/retry contracts remain release obligations. See First Customer and Limitations.

Upgrade boundaries

The package-version bump itself does not alter storage or peer-protocol formats. Do not infer rolling mixed-version compatibility from package numbers: regional routing and causal identity changes have their own protocol/snapshot contracts. Use the production runbook, take and verify a backup, and qualify the exact candidate before upgrading a live cluster.

Installation

patinaDB ships as a Docker image whose entrypoint is patinadb-raft, the Raft-replicated server binary (REST + Bolt). You do not need Rust, a compiler, or any build tooling — pull the image and run it. The image is published for both linux/amd64 and linux/arm64 — the same tag resolves to the right architecture on either platform.

Run the server

docker run -p 7687:7687 -p 21001:21001 \
  -v patinadb-data:/data \
  -e PATINADB_AUTH_PASSWORD=change-me \
  patinadb/patinadb

This starts a single self-leading node (a “cluster” of one — no peers needed) and exposes:

• REST on port 21001 — POST /cypher
• Bolt on port 7687 — for Neo4j drivers and the Neo4j Browser

/data is where the graph, engram history, and node state live — mount a named volume or bind mount so data survives a container restart.

By default the server refuses to start with no password set, so you don’t accidentally run an open database — see Authentication & TLS for how to configure it properly. For a disposable local try-out only (never expose this):

docker run -p 7687:7687 -p 21001:21001 patinadb/patinadb --insecure-disable-auth

Query it once it’s up:

curl -s -u neo4j:change-me -X POST localhost:21001/cypher \
  -H 'content-type: application/json' \
  -d '{"query": "CREATE (n:Person {name: \"Ada\"}) RETURN n"}'

See Quick Start for a walkthrough, and Configuration Reference for every server flag (--id, --addr, --db, --bootstrap, --bolt-addr, and the rest).

Next steps

• Quick Start — a five-minute walkthrough.
• High Availability — add peers for automatic failover.
• Authentication & TLS — lock the server down before exposing it.
• Editions & Limits — what Community mode includes, and how to apply a license.

Quick Start

A five-minute path to a running node and your first query.

Start the server

Start a single self-leading node (quorum of 1 — no peers needed):

docker run -p 21001:21001 -p 7687:7687 -v "$PWD/serverdata:/data" \
  patinadb/patinadb --auth-user neo4j --auth-password secret

(Or run the patinadb-raft binary directly the same way if you have it installed on the host: patinadb-raft --id 1 --addr 127.0.0.1:21001 --db ./serverdata --bootstrap --auth-user neo4j --auth-password secret.)

This exposes:

• REST on 127.0.0.1:21001 — POST /cypher
• Bolt on 127.0.0.1:7687 — for Neo4j drivers and the Browser

Query over REST:

curl -s -u neo4j:secret -X POST 127.0.0.1:21001/cypher \
  -H 'content-type: application/json' \
  -d '{"query": "CREATE (n:Person {name: \"Ada\"}) RETURN n"}'

Connect the Neo4j Browser (browser.neo4j.io) to bolt://localhost:7687 with user neo4j / password secret, and run Cypher interactively — including graph visualisation of returned nodes and paths.

Connect an official driver (Python shown):

from neo4j import GraphDatabase
drv = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "secret"))
with drv.session() as s:
    for rec in s.run("MATCH (n:Person) RETURN n.name AS name"):
        print(rec["name"])

From here:

• Add nodes for High Availability.
• Host more than one graph with Multi-Database.
• Lock it down with Authentication & TLS.

Cookbook: From Zero to Graph Analytics

This chapter is a single end-to-end walk-through: load → enforce → query → analyze → export → integrate, ending with the versioning and provenance features that make patinaDB distinct. Every command here runs against a live patinaDB server (Installation, Quick Start) over its REST endpoint (curl) or Bolt endpoint (a Neo4j driver / cypher-shell) — pick whichever fits your workflow; both run the identical Cypher.

We’ll build a tiny social graph: Person nodes connected by FOLLOWS relationships. REST examples assume a node reachable at localhost:21001 with Basic auth neo4j:secret (see Quick Start).

1. Load data

For a one-off graph, just CREATE nodes with a query:

curl -s -u neo4j:secret -X POST localhost:21001/cypher \
  -H 'content-type: application/json' \
  -d '{"query": "CREATE (u1:Person {name:\"Ada\", age:36, city:\"London\"})"}'

For a real dataset, use LOAD CSV — a streaming CSV row source in the reading pipeline, just like UNWIND but from a file. Point it at a file the server can read (see file-I/O sandboxing below):

LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CREATE (:Person {
  name: row.name, age: toInteger(row.age), city: row.city
})

# people.csv
name,age,city
Ada,36,London
Grace,40,New York
Linus,29,Helsinki
Margaret,33,Boston

LOAD CSV WITH HEADERS FROM 'file:///data/follows.csv' AS row
MATCH (a:Person {name: row.from}), (b:Person {name: row.to})
CREATE (a)-[:FOLLOWS {since: toInteger(row.since)}]->(b)

• WITH HEADERS makes each row a map keyed by header (row.name); without it each row is a list of string cells (row[0]).
• Cells are strings — coerce with toInteger / toFloat / toBoolean. A coercion may sit directly in a CREATE/MERGE pattern property, as above.
• Only file:// and bare/relative paths are read; http(s):// is rejected.

Loading online, in batches — a lot of rows at once

A bare LOAD CSV … CREATE buffers all resolved writes into one transaction (one Raft entry) before committing — fine for a small file, but an O(rows) memory cliff for a large one. Wrap the write in CALL { … } IN TRANSACTIONS OF n ROWS to chunk it into many small, independently-committed transactions instead:

LOAD CSV WITH HEADERS FROM 'file:///data/accounts.csv' AS row
CALL {
  WITH row
  CREATE (:Account {id: toInteger(row.id), name: row.name})
} IN TRANSACTIONS OF 1000 ROWS

Each batch of 1000 rows commits as its own engram (memory stays O(batch), not O(file)) and, on a cluster, as its own Raft entry. ON ERROR {CONTINUE|BREAK|FAIL} controls what happens to a failing batch. See Bulk Loading & Import.

Loading a lot of rows — bulk upsert

For loading (or reconciling) a large dataset from a file already sitting on the server’s disk, POST /mgmt/upsert (admin-only) resolves and applies a whole columnar file as one match-or-create changeset, index-accelerated and chunkable — see Bulk Upsert.

2. Enforce schema

Add constraints so bad data is rejected at write time. patinaDB supports IS UNIQUE, IS NOT NULL, IS NODE KEY, and IS :: <TYPE> on nodes, plus the same set on relationships — see Constraints for the full list:

CREATE CONSTRAINT person_name FOR (p:Person) REQUIRE p.name IS UNIQUE

status: 'Constraint created on :Person(name) IS UNIQUE'

Creating a constraint over already-populated data fails loudly if the data already violates it. Once in place, a violating write is rejected:

CREATE (p:Person {name: 'Ada'})

error: Unique constraint violation on :Person(name): value already exists

SHOW CONSTRAINTS lists them. Constraint DDL replicates to every node and rides Raft snapshots. See the Cypher chapter for the full DDL grammar.

3. Query

Ordinary Cypher — MATCH, WHERE, ORDER BY, aggregation. List people by age:

MATCH (p:Person) RETURN p.name AS name, p.age AS age ORDER BY p.age DESC

Who has the most followers? A traversal plus a count(*) group-by:

MATCH (:Person)-[:FOLLOWS]->(t:Person)
RETURN t.name AS name, count(*) AS followers
ORDER BY followers DESC, name

Over REST, the response’s scalars block is the clean column view:

"scalars": {
  "name": ["Ada", "Linus", "Grace"],
  "followers": [2, 2, 1]
}

patinaDB passes ~97% of the openCypher TCK; see Cypher Support for the exact coverage and the Limitations chapter for the gaps.

4. Analyze

Built-in, read-only graph algorithms run over an in-memory snapshot of the current graph and yield real nodes plus a per-node result. Rank the influential accounts with PageRank:

CALL patinadb.algo.pageRank('Person','FOLLOWS') YIELD node, score
RETURN node.name AS name, round(score*1000)/1000 AS score
ORDER BY score DESC

"scalars": {
  "name":  ["Ada", "Linus", "Grace", "Margaret"],
  "score": [0.387, 0.374, 0.202, 0.038]
}

patinadb.algo.wcc (weakly-connected components), patinadb.algo.degree (degree centrality), patinadb.algo.betweenness, patinadb.algo.closeness, patinadb.algo.triangleCount, and patinadb.algo.labelPropagation round out the set (seven algorithms in all), with gds.*.stream aliases for tooling. See Procedures.

To understand the shape of your data before writing queries, inspect the planner’s statistics catalog — per-property count, distinct values (NDV), and min/max:

CALL patinadb.stats('Person') YIELD label, property, count, ndv, min, max
RETURN property, count, ndv, min, max ORDER BY ndv

"scalars": {
  "property": ["age", "city", "name", null],
  "count":    [4, 4, 4, 4],
  "ndv":      [4, 4, 4, null],
  "min":      [29, "Boston", "Ada", null],
  "max":      [40, "New York", "Margaret", null]
}

The same catalog feeds cost-based entry-point selection and join ordering — it only ever changes which plan runs, never the result. See Procedures → Statistics catalog.

5. Export

Read the whole graph back out, read-only, into import-compatible files — one node file per label, one relationship file per type. GET /mgmt/export streams a tar archive:

curl -s -u neo4j:secret 'localhost:21001/mgmt/export?db=default&format=csv' \
  -o dump.tar
tar xf dump.tar

$ ls
nodes_Person.csv  rels_FOLLOWS.csv  _schema.json

$ cat nodes_Person.csv
:ID(uuid),age:int,city,name,:LABEL
1ac09593-6a05-5203-b12a-91294189c587,40,New York,Grace,Person
...

?format=parquet / ?format=arrow write the columnar equivalents. Selective exports are also available from within a query: CALL patinadb.export.csv(label, path) or CALL patinadb.export.query(query, file) write a file on the server — see Procedures → CSV export and the file-I/O sandbox note below.

Server file-I/O sandbox

LOAD CSV FROM 'file://…' and the export procedures both touch the server’s own filesystem — they read/write a file on the machine running the query, not on your client. Both are deny-by-default: an operator must explicitly whitelist directories with --allow-csv-dir/--allow-export-dir, and any query doing file I/O requires the global Admin role. See Authentication & TLS → Cypher-driven file I/O.

6. Integrate

• The server speaks Bolt, so the official Neo4j drivers and the Neo4j Browser point straight at patinaDB.
• Change Streams (CDC) (GET /changes) let an external system react to graph changes as they happen — cache invalidation, search-index sync, ETL.
• The full REST API and management endpoints (/mgmt/*) cover backup/restore, cluster administration, and metrics.

7. Time-travel & provenance — the differentiators

Every write is recorded as an engram — a git-like commit. Build up some history:

CREATE (a:Person {name:'Ada', team:'core'})
CREATE (g:Person {name:'Grace', team:'core'})

MATCH (a:Person {name:'Ada'}) SET a.team = 'infra'

List the history (oldest first) with CALL patinadb.engrams(), then view a git-style diff of any single change:

CALL patinadb.diff('<set-engram-id>')

engram ae2eae1a-3fcc-5c26-aad9-b11c4949b5c3
parent:    54f17ad5-bc1d-5686-867a-9891bc906d92
timestamp: 1783448468

~ (7530d903:Person).team: 'core' -> 'infra'

Query the graph as it was at any past engram — no audit table required. Prefix the read with USE <db> AS OF '<engram-id>' (or pass an at field over REST):

USE default AS OF '<prev-engram-id>'
MATCH (a:Person {name:'Ada'}) RETURN a.team AS team
-- team: 'core'   (it is 'infra' at HEAD)

See Time Travel, Diffs, and Engrams.

Finally, Anamnesis auto-projects that change stream into a queryable W3C-PROV graph in a companion <db>__anamnesis database — who, what, when, and optional source / confidence / derived_from — enriched straight from Neo4j transaction metadata. It is opt-in per database (CALL patinadb.anamnesis.enable()). Where other databases leave you to assemble provenance from triggers or CDC, here it ships in the engine.

Where to go next

• Cypher Support — the full query language and its coverage.
• The Server (Raft) — run patinaDB as a replicated service.
• Full-Text & Vector Search — search in the engine.
• Limitations — read this before a production workload.

Data Model

patinaDB stores a labelled property graph.

Vertices (nodes)

A vertex has:

• A stable UUID identity (assigned on creation, baked into the engram log so replays are deterministic).
• A primary label plus zero or more secondary labels (multi-label nodes are supported; labels(n) returns all of them).
• A set of properties — string keys mapped to typed values.

CREATE (a:Person:Employee {name: 'Ada', born: 1815})

Edges (relationships)

An edge is directed, has exactly one type (a label), connects two vertices, and may carry its own properties:

CREATE (a)-[:KNEW {since: 1833}]->(b)

Edges can be traversed in either direction, and patterns may be left-to-right, right-to-left, or undirected (-[:KNEW]-), in which case both directions are searched.

Element identity (id() / elementId())

• elementId(n) is the blessed, stable string handle for a node — the vertex’s UUID as a string. It is exactly what a driver stores and what you look nodes back up by:

  MATCH (n:Person) WHERE elementId(n) = $eid RETURN n
  

• id(n) returns the same UUID (as a Uuid). A lookup by the UUID string resolves the node on every surface — WHERE id(n) = '<uuid>', WHERE elementId(n) = '<uuid>', and MATCH (n {__id: '<uuid>'}) — so a round-trip never silently returns zero rows. All three are served by an O(1) point lookup, not a scan.

  There is no integer node id. patinaDB has no Neo4j-style integer id, so WHERE id(n) = <number> (or a boolean) raises a clear error rather than silently matching nothing — compare against a string element id instead.

• id is an ordinary property — patinaDB does not hijack it. You may store and query your own id freely, exactly as in Neo4j:

  CREATE (:User {id: '550e8400-e29b-41d4-a716-446655440000', name: 'ada'})
  MATCH  (u:User {id: '550e8400-e29b-41d4-a716-446655440000'}) RETURN u  // matches
  MATCH  (u:User) WHERE u.id = 42 RETURN u                               // a plain property
  

  The reserved property name __id is the internal element id: n.__id reads the node’s UUID and MATCH (n:Label {__id: <uuid>}) is the O(1) point lookup on it. Property names beginning with __ are reserved for patinaDB; writing __id — via a pattern property, SET n.__id = …, SET n = {…} / n += {…}, or bulk import — is rejected with a Neo.ClientError.Statement.ArgumentError and leaves the graph untouched.

• id(r) / elementId(r) return a relationship’s canonical <start>:<end>:<type> string handle. Because an edge is that triple (the storage index keys on it), the handle is unique and stable. Fetch a relationship by it with a scan:

  MATCH ()-[r:KNEW]->() WHERE elementId(r) = $handle RETURN r
  

  A driver’s r.element_id over Bolt 5+ carries the identical handle. (The legacy Bolt integer id field is derived from a lossy 63-bit hash and is not round-trippable — always use element_id.) The model is id-less by design: it cannot store two parallel edges of the same type between the same pair of endpoints, so the handle is unique over everything it can represent.

Parallel edges: what actually happens

The “no parallel edges” rule above is a storage fact — the triple (start, end, type) is the edge’s identity. Unlike Neo4j, an edge here has no independent relationship identity of its own, so a second edge of the same type between the same two nodes has no separate place to live.

A CREATE that would collide with an existing edge is therefore rejected:

CREATE (a)-[:R {n: 2}]->(b)   // (a)-[:R {n: 1}]->(b) already exists

fails with Neo.ClientError.Schema.ConstraintValidationFailed. The rejection happens before anything is written: the existing edge keeps its properties (n: 1 above), the graph is left untouched, and no write counters are reported. There is no partial effect to clean up.

This is deliberate. CREATE is never MERGE semantics in Cypher, and the alternative — quietly overwriting the existing edge’s properties — loses data without saying so. If you want the upsert, say so explicitly with MERGE.

Bulk import paths (LOAD CSV, CSV/Parquet/Arrow import) are not covered by this rejection. A bulk re-import is often deliberately upsert-shaped, so those paths keep the overwrite behaviour.

Modelling consequence. If your domain genuinely has several relationships of the same kind between the same two nodes — a user commenting on the same ticket twice, two separate employments of one person at one company — an edge per occurrence cannot express it. Reify the occurrence as an intermediate node instead:

// NOT this — the second CREATE is rejected, so the second comment is lost:
//   (u:User)-[:COMMENTED]->(t:Ticket)

// This — one node per comment, edges stay unique:
CREATE (u:User)-[:WROTE]->(c:Comment {id: $commentId, body: $body, at: datetime()})
CREATE (c)-[:ON]->(t:Ticket)

A UNIQUE constraint on the intermediate node’s business key (:Comment(id) above) then gives you real duplicate protection on top — see Constraints.

Property value types

The AttributeValue type covers:

Three-valued (Kleene) logic

Comparisons and boolean operators follow SQL-style three-valued logic. Any comparison involving null yields Unknown, not true or false:

RETURN 1 = null        // null (Unknown)
RETURN null AND false  // false  (false dominates)
RETURN null OR true    // true   (true dominates)

IS NULL / IS NOT NULL remain strictly two-valued, by specification. In a WHERE filter, Unknown collapses to “not matched”.

Storage internals (informational)

The on-disk tables, key layouts, inline labels and property storage are described in How Storage Works.

Properties are kept in a label-scoped, order-preserving value index, so a scan over :Person.born never returns :Robot nodes with the same value, and sorted pagination over a property is O(limit) rather than a full scan. Compound indexes over several fields accelerate equality-prefix + sort queries. You normally don’t manage these directly — they are maintained automatically on writes. To declare your own indexes and enforce data integrity, see Constraints and Query Planning; Limitations lists the current DDL gaps.

Cypher Support

patinaDB implements a subset of Cypher, not complete Neo4j/openCypher parity. The openCypher Compatibility Reference is the feature-by-feature inventory, including unsupported syntax, function names and semantic differences. The openCypher Technology Compatibility Kit (TCK) is a prioritized gap report, not a strict regression gate or certification. This chapter provides an overview and usage examples.

Reading

• MATCH with node/edge patterns, optional labels, property maps, and all three directions (->, <-, - undirected).
• OPTIONAL MATCH.
• Variable-length paths — -[:T*1..3]->, [*0..n] (length-0 self-row), shortestPath.
• WHERE — full boolean expressions: AND / OR / NOT / XOR, comparisons, IN, IS NULL, STARTS WITH / ENDS WITH / CONTAINS, regex =~, nested parenthesisation.
• RETURN with DISTINCT, aliases (AS), expressions, ordering.
• WITH chained query stages (projection, filtering, aggregation hand-off, ordering before projection).
• UNWIND, UNION / UNION ALL.
• ORDER BY / SKIP / LIMIT — single-key ordering uses an index fast-path; multi-key falls back to a post-sort.

Relationship type alternatives in paths

A MATCH path such as (a)-[:ROAD|RAIL*0..3]->(b) accepts either listed relationship type at each hop, including routes mixing both types. Repeating a type name does not duplicate a route. Distinct routes remain distinct result rows unless the query requests DISTINCT; matching endpoints alone do not collapse them. A physical relationship cannot be reused within one matched path. A lower bound of zero permits the empty path where the endpoint constraints match.

The same type alternatives work with incoming and undirected patterns. CREATE and MERGE still require a single fixed relationship type and reject variable-length relationship creation.

Named shortest paths

MATCH p = shortestPath((a)-[:ROAD*]->(b)) selects one unweighted shortest path per endpoint pair. allShortestPaths selects every tied shortest path; it does not enumerate longer paths. When several shortest paths tie, shortestPath does not promise which one is chosen.

The supported named form contains one relationship segment, with a minimum length of zero or one and an optional maximum. Incoming, outgoing and undirected traversal and relationship-type alternatives are supported. A zero minimum permits an empty path for matching endpoints. Identical endpoints with a positive minimum raise an error. Disconnected pairs produce no path; OPTIONAL MATCH retains its null-row behavior. This zero-length behavior differs from Neo4j with its default dbms.cypher.forbid_shortestpath_common_nodes=true, which rejects common endpoints even when the minimum is zero.

Relationship-variable bindings, relationship property maps, multiple segments, and a minimum above one are rejected in this form. A WHERE attached to the shortest-path MATCH and another relationship pattern in that same MATCH clause are also rejected. Filtering after WITH p is supported: it filters paths already selected and does not search for a longer path satisfying the filter.

Shortest-path search uses the query’s execution-memory budget and cancellation checks. Search buffers and emitted path collections also observe PATINADB_MAX_AGG_ROWS. A refusal aborts the query; it does not return an incomplete set of tied shortest paths as a successful result. Result memory is conservatively retained for the query-result lifetime, so discarded intermediates can cause an early budget refusal.

Membership values

Relationship patterns can be Boolean operands in projections, for example RETURN NOT ((a)-[:LINK]->(b)) AS disconnected or RETURN ((a)-[:LINK]->(b)) AND true AS connected. Named pattern bindings must already exist in scope. A bare pattern is not a general scalar value: returning (a)-[:LINK]->(b) alone, assigning it directly, or passing it to size() is rejected. Use an explicit EXISTS { (a)-[:LINK]->(b) } for a standalone Boolean existence result. With a null endpoint there is no matching path, so the negated existence result is true. Scalar parentheses and list expressions keep their ordinary meaning.

item IN items and item NOT IN items compare the value bound to item. For example, a node tested against collect(node) remains an entity; it is not implicitly converted to its UUID. Scalar and list bindings likewise retain their types. Use an explicit property such as node.id, or node.__id for the internal UUID, when the list contains those values. Dotted property membership keeps its existing lookup and seek behavior; null membership retains three-valued logic.

The bare-binding parser correction in PR #1410 has focused and core regression coverage. It does not fix the existing identifier limitation for literal dots inside backticks. No query performance claim follows from this semantic correction.

Loading CSV

LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
WITH toInteger(row.id) AS id, row.name AS name
CREATE (:Person {id: id, name: name})

LOAD CSV [WITH HEADERS] FROM '<url>' AS <var> [FIELDTERMINATOR '<c>'] streams rows from a CSV file into the query as a row source — like UNWIND, but from a file. It is read row by row (the file is never fully buffered).

• WITH HEADERS → each row is a map keyed by the header, so row.columnName works. Without it, each row is a list of string cells: row[0], row[1], ….
• Cells are strings (openCypher semantics). Convert explicitly with toInteger / toFloat / toBoolean. A coercion may sit directly inside a CREATE/MERGE pattern property — CREATE (… {p: toInteger(row.x)}) — as well as in a WITH; only a row[i] list-index expression still needs a WITH stage first.
• FIELDTERMINATOR '<c>' overrides the delimiter (default ,).
• Sources: file:///absolute/path, file://host/absolute/path (the host is ignored), and bare/relative filesystem paths. Quoted fields and embedded delimiters/newlines are handled. http(s):// URLs are not yet supported.

Large loads (memory): a plain LOAD CSV … CREATE buffers all its writes into one transaction (on the server, one Raft entry), so a very large load has an O(rows) memory cliff. Wrap it in CALL { … } IN TRANSACTIONS (below) to chunk it into many small commits — the online answer. For a large columnar file already on the server, POST /mgmt/upsert is faster still — see Bulk Upsert. A LOAD CSV … RETURN row read query streams end-to-end.

Server-side file reads (security): on the server, LOAD CSV FROM 'file://…' reads a server-side file, so it is deny-by-default: it reads only directories whitelisted with --allow-csv-dir, and any file-I/O query is raised to require the global Admin role. See Bulk Loading → Server file-I/O sandbox and Authentication & TLS.

Batched writes: CALL { … } IN TRANSACTIONS

LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CALL {
  WITH row
  CREATE (:Person {id: toInteger(row.id), name: row.name})
} IN TRANSACTIONS OF 1000 ROWS

CALL { <write-subquery> } IN TRANSACTIONS [OF <n> ROW[S]] [ON ERROR {CONTINUE|BREAK|FAIL}] chunks a large write into many small, independently committed transactions instead of one giant one. The outer stream (typically LOAD CSV … AS row or UNWIND $rows AS r) feeds rows to the inner subquery, which runs and commits every n rows as a separate transaction (default n = 1000). This bounds peak memory to one chunk — the fix for the LOAD CSV … CREATE RAM cliff — and, on a cluster, replicates as one small entry per chunk instead of one unbounded one.

• Each chunk is its own commit ⇒ its own engram (so batched ingest is versioned and time-travellable, unlike the offline importer), and partial progress is durable: a crash between chunks recovers to a chunk boundary (all-or-nothing per chunk). A later chunk sees the writes committed by earlier chunks.
• OF <n> ROW[S] sets the chunk size (omit for the 1000 default).
• ON ERROR — FAIL (default) aborts the whole statement on a failing chunk; CONTINUE skips the failing chunk and commits the rest; BREAK stops after the failing chunk, keeping the chunks already committed. A skipped/failed chunk changes nothing (it is resolved against a throwaway copy of the graph, so a partial chunk is never left behind).
• The CALL { … } IN TRANSACTIONS must be the final clause of the query.

Writing

• CREATE, MERGE (match-or-create), SET (properties, map merge, and SET n:Label label mutation), REMOVE, DELETE / DETACH DELETE.
• FOREACH.
• Writes inside WITH stages.

Expressions & functions

• Arithmetic, string, list, and map operators; list/map indexing and slicing.
• Aggregation — count (incl. count(*)), sum, avg, min, max, collect, with GROUP BY semantics (mix of aggregated and grouping keys in RETURN), DISTINCT inside aggregates, and aggregate hoisting through nested function calls and arithmetic.
• CASE (simple and searched forms). A simple CASE value may be a complete Boolean expression, for example CASE n.active IS NULL WHEN true THEN .... Boolean values use the same equality and NULL behavior as other simple CASE values; CASE WHEN predicate THEN ... remains the searched form.
• List comprehensions and pattern comprehensions.
• Quantifiers — any / all / none / single, including aggregate and count(*) sources.
• Scalar functions — toInteger, toString, size, labels, type, nodes, relationships, range, and many more.
• Temporal functions — date, time, localtime, datetime, localdatetime, duration, plus namespaced calls (date.truncate, duration.between, datetime.fromepoch) with ISO-8601 week numbering.

datetime({epochMillis: value}) constructs an instant from integer Unix milliseconds, including negative values. For example, datetime({epochMillis: -1}).millisecond is 999. An optional timezone changes the local representation of that instant; calendar components can override it. A no-op override in an ambiguous daylight-saving hour retains the original offset. Invalid epoch types or competing temporal selectors raise an argument error; datetime(null) still returns null.

Epoch-map construction rejects instants or override years outside the supported calendar range rather than wrapping them. Named zones have an additional calendar-library range bound. A calendar override into a nonexistent local daylight-saving time raises an error; it does not shift the time forward as Neo4j does. This is not full temporal-range or override compatibility.

Value semantics

Numeric equivalence

patinaDB treats an integer and a mathematically-equal float as the same value for equivalence purposes, matching Neo4j. The fold is exact: a float counts as its integer twin only when it is finite, has a zero fractional part, and round-trips bit-for-bit ((f as i64) as f64 == f) — so a large integer that is not exactly float-representable never falsely merges, and two genuinely different integers never collide.

Because of this, DISTINCT, an implicit GROUP BY key, and a distinct UNION all merge numeric twins:

UNWIND [1, 1.0] AS x RETURN DISTINCT x
-- 1 row (previously 2)

UNWIND [1, 1.0, 2] AS x RETURN x, count(*)
-- 1 → 2, 2 → 1     (1 and 1.0 fall into one group)

The surviving value keeps its original type — first occurrence wins (as in Neo4j), so the result shows whichever of 1 / 1.0 appeared first, not a normalized form. UNION ALL is unaffected (it never de-duplicates). The same equivalence drives numeric-twin-aware UNIQUE constraints.

Cross-type ORDER BY

A missing sort property is NULL. A storage error while reading a required node or relationship sort property fails the query, for both full sorting and ORDER BY ... LIMIT. Empty inputs and LIMIT 0 do not require those reads.

When a single ORDER BY column holds values of different types, patinaDB follows Neo4j’s documented precedence, lowest to highest:

Map < Node < Relationship < List < Path < Temporal
    < String < Boolean < Number < Point < Polygon < NaN < Null

Every temporal type (Date, Time, LocalTime, DateTime, LocalDateTime, Duration) now sorts between Path and String — i.e. before strings and numbers (previously they sorted after numbers). min() and max() follow the same order. This applies only to a mixed-type column; a column of a single type is ordered normally and is unaffected. (Point and Polygon are patinaDB spatial extensions beyond Neo4j’s list, shown here for completeness.)

Arithmetic errors

Integer division and modulo by zero raise Neo.ClientError.Statement.ArithmeticError. Integer +, -, *, and sum() over integers raise the same error on i64 overflow, rather than silently saturating or wrapping. A mixed integer/float expression is promoted to float before the operation (so it never overflows there — Neo4j parity), and float arithmetic is unchanged: it still yields ±Inf / NaN, not an error.

Procedures

CALL invokes built-in procedures (history, diff, full-text/vector search, graph algorithms, statistics, export, cache stats). See Procedures. User-defined scripting-language procedures are not supported, but user-defined read-only WASM procedures are — see WASM Procedures.

DDL

• CREATE INDEX / DROP INDEX / SHOW INDEXES — single-property and compound (multi-field) B-tree indexes. SHOW INDEXES returns an indexes list of strings. Single-property entries remain bare property names and point indexes retain their POINT FOR … form. Each registered compound definition appears once, for example COMPOUND INDEX [`batch_lookup`] FOR (:`Batch`) ON (`run_id`, `id`). The field order is part of its identity; matching registered names are listed together. Unnamed definitions omit the name list. Identifiers are backtick quoted, with embedded backticks doubled. This inventory reports registered schema; it does not report a separate readiness or population state.

• CREATE CONSTRAINT / DROP CONSTRAINT / SHOW CONSTRAINTS — all four Neo4j constraint kinds (uniqueness, existence, node-key/relationship-key, property-type), on both nodes and relationships. See Constraints.

• CREATE FUNCTION … LANGUAGE wasm / DROP FUNCTION / SHOW FUNCTIONS — user-defined, sandboxed, read-only WASM procedures. See WASM Procedures.

• CREATE POINT INDEX / DROP POINT INDEX / SHOW POINT INDEXES — accelerates spatial radius/bbox/kNN queries. See Spatial / Geo.

• CREATE EDGE SORTED INDEX (patinaDB extension) — pre-orders, per anchor vertex, that anchor’s targets over one relationship type by a target property, so a traversal-fan-out + ORDER BY target.prop [DESC] LIMIT k is served by a seek + take (no fan-out, no post-sort — bench Q5–Q8). Two orientations:

  CREATE EDGE SORTED INDEX [name] FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at
  CREATE EDGE SORTED INDEX [name] FOR (m:Ticket)<-[:HAS_LABEL]-()  ON m.created_at
  SHOW EDGE SORTED INDEXES
  

  Creating the index over a populated graph backfills existing edges; it is kept live on writes. EXPLAIN names limit.edge_sorted_topk once a covering index serves the query. On a server it replicates to every node (deterministic re-run). The name is optional and cosmetic (the def is identified by its shape). Not yet carried in Raft snapshots — see Limitations.

• CREATE FULLTEXT INDEX / DROP INDEX / SHOW FULLTEXT INDEXES — see Full-Text Search.

• CREATE DATABASE / DROP DATABASE / SHOW DATABASES (server only) — see Multi-Database.

• EXPLAIN and PROFILE render the physical operator tree.

Query introspection

EXPLAIN MATCH (p:Person)-[:KNEW]->(q) WHERE p.born < 1850 RETURN q
PROFILE MATCH (p:Person {name: 'Ada'}) RETURN p

EXPLAIN also includes a Decisions: section after the physical-access details. Each entry names the decision point, chosen alternative, available row/cost estimates, reason and provenance. These are the selector’s existing estimates, not measured runtimes. Priority walks stop when their winner is known, so later alternatives may be absent.

PROFILE adds Decisions (executed): and an Executed operators (query thread): section. Operator numbers are preorder indices of the executed plan. rows=N counts successful rows actually pulled from that iterator; a LIMIT can leave an input only partly consumed. MatchScan also shows its entry estimate and q-error (the larger of estimate/actual and actual/estimate, replacing zero with one). An operator bypassed by a fast path is marked rows=unobserved (bypassed/fused). Worker internals and mutation/seeded paths that bypass the iterator are not per-operator measurements. The separately planned access tree is still labelled Planned access (before execution); use the executed section for row counts.

The audit retains at most 128 distinct decisions and coalesces repeats by point, subject and chosen alternative. repeats includes repeated seeded executions; the displayed estimates are from the first observation. Ordinary queries do not construct audit events or use a per-row counting adaptor.

EXPLAIN shows the chosen plan (scan strategy: compound index, property-value index, label scan, or all-vertices) without running it; PROFILE runs and annotates it.

Planning note for filtered pages

For indexed keyset pagination, use a matching compound index and a qualified ordering property, for example:

CREATE INDEX tickets_by_project_id IF NOT EXISTS
FOR (ticket:Ticket) ON (ticket.project, ticket.id)

MATCH (ticket:Ticket {project: $project})
WHERE ticket.id > $after
RETURN ticket.id AS id, ticket.title AS title
ORDER BY ticket.id LIMIT $limit

Use separate parameterized query shapes when an optional filter/cursor is absent, rather than assuming ($after IS NULL OR ticket.id > $after) preserves a seek. ORDER BY id is valid syntax, but the #1261 million-ticket application diagnosis observed a separate sort for that projected alias while ORDER BY ticket.id selected the compound sort-covered path. Check EXPLAIN on the actual query; equivalent result semantics do not guarantee identical access plans. The local engineering evidence is docs/customer-blackbox-million-1261-2026-09-16.md.

For a required single-node match with equality predicates covering exactly a compound index’s fields, count(*) or count(ticket) can use agg.filtered_count_scan: it counts matching index entries rather than loading each node. This is still a scan of matching entries, not a precomputed counter. Optional matches, additional filters, secondary-label matches and unsupported value encodings retain the general execution path. EXPLAIN reports the actual choice. The snapshot transaction and numeric-equality semantics are unchanged.

A non-null scalar comparison on the ordered property can prove that missing values cannot match, avoiding a separate whole-label population check for that compound sort. Without such a predicate, NULL placement must still be preserved; first-page planning can therefore cost more than a bounded keyset page. These planning rules do not promise a fixed latency for arbitrary distributions.

What is not supported

A precise list lives in Limitations. The headline gaps: no general scripting-language user-defined procedures (only sandboxed read-only WASM procedures), no triggers, and partial subquery, expression and error-semantic compatibility. See the compatibility reference rather than using a historical TCK percentage as a current support contract. (Index DDL and all four constraint kinds, on nodes and relationships, are supported; see Constraints and Limitations for the exact scope.) Side-effect counters (nodes_created, relationships_created, properties_set, … in query summaries) are reported on both surfaces: as stats in the REST write response and as ResultSummary.counters over Bolt — see Bolt → Side-effect counters for which counters exist and which are deliberately never emitted.

openCypher Compatibility Reference

This is the support inventory for the 0.15.1 source tree, reviewed on 2026-09-09 at 6fb6d521. It describes the implemented language, not a claim of complete openCypher, Neo4j Cypher 5/25, GQL, APOC or GDS compatibility. Bolt connectivity and neo4j:// routing do not imply language compatibility.

Supported below means that the stated form is implemented; it does not mean that every combination, invalid input or error code matches Neo4j. Partial means that the boundary matters to application correctness. Unsupported means that applications must not use that form. Parsing successfully is not sufficient evidence of equivalent execution, especially for the subquery restrictions below.

Use this chapter for porting decisions, Cypher Support for examples, and Limitations for storage and operational differences.

Clauses and graph patterns

Why labels and types are static

Not supporting :$(expression) is a deliberate design decision, not an undocumented compatibility promise. A static label/type gives the planner a known target for label/type-specific access paths and eligible property-index seeks, rather than requiring runtime dispatch across varying labels/types. It does not guarantee an index seek for every query; indexes, predicates and planner decisions still matter. This is the design rationale, not a measured performance comparison against dynamic Cypher in another database.

For mixed-label/type input, group items in the client by their label/type and submit one static query template per group, using UNWIND $rows for values. Select templates from an application-controlled allowlist; never interpolate untrusted labels/types into query text. Grouping into separate commits changes atomicity; use an explicit transaction when the supported operations must commit together. This keeps runtime variability in parameter values, not graph schema identifiers.

Subqueries and procedures

For batching, file access, explicit Bolt transaction restrictions and partial commit handling, consult Bulk Loading, Cypher Support, and Bolt. Do not assume batching also bounds every outer-query sort, aggregation or row materialization.

Expressions and values

Function arguments support postfix list slices/indexes and map-field access: RETURN size([1,2,3][1..]) returns 2, and RETURN abs({nested: {value:-7}}.nested.value) returns 7. Omitted and negative slice bounds use the same expression semantics as an intermediate WITH value. This does not imply full Neo4j expression compatibility in every grammar position; test the exact expression when porting a query.

Built-in scalar function inventory

These are the function names registered by the core grammar/evaluator. A name here does not imply every Neo4j overload, coercion or invalid-input error is implemented. Unknown scalar names are not a plugin discovery mechanism.

range(start, end [, step]) includes reachable endpoints and stops before the next step would overflow a signed 64-bit integer. A zero step is rejected; materialized results remain subject to the configured aggregate-row limit.

Not present as core scalar functions: randomUUID, log10, e, cot, haversin, char_length, character_length, normalize, valueType, toIntegerList/toFloatList/toBooleanList/toStringList, and arbitrary apoc.* functions. This is an explicit inventory, not “and all other Neo4j functions.” Named procedure equivalents, where documented, use CALL instead.

Vector similarity: supported without an index

RETURN vector.similarity.cosine([1,0], [0,1]) AS score returns 0.5. This is normalized cosine (1 + raw cosine) / 2, not raw cosine: identical nonzero vectors score 1, orthogonal vectors 0.5, opposite vectors 0. vector.similarity.euclidean([0,0], [3,4]) returns approximately 1/26, using 1 / (1 + squared distance).

Both are scalar functions over equal-length numeric lists, not CALL procedures and not nearest-neighbour searches. They require no vector index. Evaluation converts components to f32; null arguments propagate null, dimension mismatches and non-numeric components error. This is not a guarantee of all Neo4j invalid-vector edge cases. Use db.index.vector.queryNodes for indexed Top-k search; see Vector Search.

Important semantic differences

• Expression membership: IN and NOT IN accept expression-valued left operands, for example split(node.issue_key, '-')[0] IN $projects after a procedure’s WITH node, score. This does not make arbitrary predicates index-seekable; simple property membership retains its existing planner path.
• Grouped predicates: a grouped WHERE predicate followed by = true is not generally accepted; use the predicate directly in WHERE.
• Identity: nodes use UUIDs; elementId(node) is the portable string handle. id(node) is not a Neo4j integer ID. Relationship identity is the endpoint/type triple. Two relationships of the same type between identical endpoints cannot coexist; Cypher duplicate creation errors, while bulk import has a separate overwrite contract. See Data Model.
• Synthetic label: an otherwise unlabelled stored node uses the primary label Node. This is not a fully label-free Neo4j storage model; the TCK comparator normalizes this difference, so TCK results do not disprove it.
• Strings: the scalar evaluator’s size(string) counts UTF-8 bytes, not Unicode characters. For example, size('é') is 2. Other string operations need not use the same unit; test non-ASCII application text explicitly.
• Nulls: the scalar evaluator returns true for isEmpty(null), not null. Do not use it as a drop-in replacement for IS NULL. Null left operands in membership tests remain unknown even for an empty list; consequently WHERE null NOT IN [] does not retain the row.
• Temporal clocks: .statement(), .transaction() and .realtime() route to the same current-time constructor family. They do not establish Neo4j’s distinct statement/transaction clock lifetimes. Extended years, timezone serialization and very large durations also need compatibility testing.
• Numeric errors: integer division/modulo by zero and checked integer arithmetic overflow raise arithmetic errors; floating-point operations can produce infinity/NaN. Integer/float equivalence for grouping and DISTINCT is documented in Value semantics.
• Error compatibility: not every invalid argument, deleted-entity access, procedure signature or semantic error has the same validation phase/status as Neo4j. A successful syntax check is not a compatibility certificate.

Schema, administration and non-openCypher extensions

These features are useful, but are separate from an openCypher language claim.

Explicit non-goals of this subset

• Modern quantified path patterns/relationships and selectors such as SHORTEST k, ALL SHORTEST, ANY, ACYCLIC or TRAIL in their newer Cypher/GQL syntax; use the documented legacy patterns instead.
• Dynamic-label/type syntax such as :$(expression); general label expressions such as (node:Ticket&Open) are not the supported :Ticket:Open form.
• INSERT, LET, FILTER, FINISH, NEXT, query USE, SQL window functions, SQL GROUP BY/HAVING, and arbitrary multi-statement scripts in one query.
• Triggers, unrestricted server-side scripting, writable WASM UDFs, and a Neo4j plugin ABI.

Evidence and maintenance

The implementation inventory is grounded in these source files:

• crates/patinadb/src/cypher/cypher.pest: accepted syntax and function names.
• cypher/parse_query.rs, parse_expr.rs, parse_ddl.rs: validation and lowering boundaries; grammar acceptance alone is not the support decision.
• plan/eval/scalars.rs, plan/eval/shortest_path.rs, plan/exec/expr_eval.rs: expression behavior and the semantic differences called out above.
• execution/in_transactions.rs and the server dispatchers: batching versus ordinary statement/transaction behavior.
• crates/patinadb/tests/manual_cypher_contract.rs: executable examples and negative controls for selected documented unsupported forms.
• crates/patinadb/tests/opencypher_tck.rs: broader conformance inventory.

The TCK runner is informational, not a strict all-green gate. Its comparison normalizes synthetic labels and some value representations; identity correlation across cells is not fully modeled. One pinned Gherkin parsing limitation also prevents a feature’s scenarios from running. Such scenarios must be counted as not passing, not silently excluded. A fresh TCK run still cannot replace this feature-by-feature contract or tests of the backend’s exact queries.

The 2026-09-09 run on the revision above reports 3755 passed, 113 executed failures, and 29 never-run scenarios: 3755 of 3897, with 142 not passing (#858 inventory). The process exits successfully even with those failures. This replaces the older 3754-pass inventory; it is not an all-green result.

Recheck after language changes:

cargo test --locked --release -p patinadb --test manual_cypher_contract
cargo test --locked --release -p patinadb --test opencypher_tck

Archive the complete TCK output, source revision and never-run scenario count. Do not publish an old pass percentage as the current release’s certification.

Counting independent optional branches

For supported independent one-hop OPTIONAL MATCH branches, count(*) can count their combinations without retaining the combined rows. Empty optional branches still contribute one null-extended row. This path can complete when the count exceeds the materialized-row limit; queries that retain those rows still use the configured limit. EXPLAIN/PROFILE identify this execution as agg.independent_optional_count. Other optional shapes retain normal execution.

Bulk Loading & Import

patinaDB supports offline CLI import and online bulk loading. The three online paths below are versioned (recorded in the engram log) and, on a cluster, replicated. Offline CLI import writes the datastore directly without engrams.

Bulk upsert (large files, match-or-create)

POST /mgmt/upsert, /mgmt/upsert-edges, and /mgmt/upsert-graph resolve and apply a whole columnar file (CSV / Parquet / Arrow) — or a coupled JSON node+edge subgraph — as one changeset, per-row match-or-create against a match key, index-accelerated. A chunk_rows field splits a very large load into many bounded, independently-committed engrams so peak memory (and Raft entry size) stays O(chunk) instead of O(rows).

curl -s -u neo4j:secret -X POST localhost:21001/mgmt/upsert \
  -H 'content-type: application/json' \
  -d '{"db":"default","file":"/srv/import/people.csv","label":"Person","match":["email"]}'

The file path is confined to the server’s filesystem sandbox (see below); this is the fastest, largest-scale online path — see Bulk Upsert for the full request shape, merge/conflict policies, and chunking guidance.

Offline CLI import

patinadb import loads node and relationship CSV files into a database while it is offline. This direct datastore operation does not create import engrams or establish a replicated history baseline. Do not use it against a running server or treat a successful current-graph read as proof of history or replica recovery. Node headers identify :ID, :LABEL, and property columns; relationship headers identify :START_ID, :END_ID, :TYPE, and property columns. A property header may declare a scalar type (name:int, active:boolean, score:double) or one of the typed-list forms below. CSV quoting follows the CSV reader’s standard quoting rules, so a JSON array cell containing commas must itself be CSV-quoted, for example:

:ID,:LABEL,tags:string[]
u1,Person,"[""red"",""blue, green""]"

For typed-list properties, the cell contains a JSON array:

An empty cell means the property is absent. [] stores an empty list, while [""] stores a one-element list containing the empty string. JSON null elements are allowed. Array order and duplicate elements are retained; string contents, including leading or trailing whitespace, are preserved. The cell must be a JSON array whose non-null elements match the declared type. Nested arrays, objects, malformed JSON, and values of another element type are rejected.

Typed-list declarations apply to CSV import; typed Arrow List columns remain unsupported. This does not establish a CSV/Arrow export-import round-trip contract.

Scalar string properties (string, char, an untyped header, or an unrecognised type suffix) preserve the CSV cell exactly, including leading and trailing whitespace. A zero-length cell means the property is absent; a whitespace-only cell is a present string containing that whitespace. Earlier importer behavior trimmed scalar strings and treated whitespace-only cells as absent. Re-importing such files with this behavior can therefore change stored scalar string values or add a previously absent whitespace-only property. This is an intentional migration behavior; scalar numeric, Boolean, date, and datetime parsing retains its prior surrounding-whitespace handling.

For a :LABEL cell, semicolon-separated labels form the complete label set: the first is primary and later unique labels are secondary. Whitespace around label names is ignored; empty names (including A;;B or A;) are rejected. A later row for the same ID replaces the previous set; a single primary label clears existing secondary labels. This also applies to Arrow/Parquet label columns. Replacement does not imply deletion of omitted properties.

The importer retains its existing durable batch and interrupted-import recovery contracts. It does not make an entire multi-file import rollback-atomic.

Online: LOAD CSV

LOAD CSV streams rows from a CSV file into a running query as a row source — like UNWIND, but from a file. It goes through the normal write path, so each load is versioned (an engram) and, on a cluster, replicated.

LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CREATE (:Person {id: toInteger(row.id), name: row.name})

• WITH HEADERS makes each row a map keyed by header (row.name); without it each row is a list of string cells (row[0]).
• Cells are strings — coerce with toInteger / toFloat / toBoolean. A coercion may sit directly in a CREATE/MERGE pattern property (as above); a row[i] list-index expression still needs a WITH stage first.
• FIELDTERMINATOR '<c>' overrides the , delimiter. http(s):// URLs are rejected; only file:// and bare/relative paths are read.

The full clause reference is in Cypher Support → Loading CSV.

The un-chunked memory cliff

A bare LOAD CSV … CREATE streams the source row by row, but buffers all resolved write operations into one transaction (on the server, one Raft entry) before committing — an O(rows) memory cliff for a large load. A LOAD CSV … RETURN row read query streams end to end and has no such cliff.

Correction (issue #1060, measured 2026-08-21): the O(rows) figure above holds for the plain example shown — a CREATE with no MATCH at all. The equally ordinary foreign-key-lookup import idiom —

LOAD CSV WITH HEADERS FROM 'file:///data/orders.csv' AS row
MATCH (c:Customer {id: toInteger(row.customer_id)})
CREATE (c)-[:PLACED]->(:Order {id: toInteger(row.order_id)})

— is not O(rows). Before the #1060 fix, LOAD CSV could never seed the planner’s index-seek restructure (issue #429/#367) the way a top-level UNWIND can, so MATCH (c:Customer {id: …}) fell back to a full label scan over every :Customer vertex, repeated once per CSV row — O(R · rows) where R is the distinct-customer count, not O(rows). Measured on a hub-with-N-leaves fixture (R = distinct parents, N = rows per parent): the real LOAD CSV path cost the same order of magnitude, ~3.9 GB peak RSS, as a synthetic double-UNWIND cross-join at R = 25, N = 20 000 — see crates/patinadb-bench/examples/existence_isolate_probe.rs (modes crossjoin_build / loadcsv_build) for the harness and crates/patinadb/src/execution/load_csv_tests.rs’s loadcsv_seed_restructure_matches_unoptimised_reference for the fix’s differential proof.

The fix makes LOAD CSV a seed candidate like UNWIND, so the shape above now seeks when the compared value is a bare CSV field (row.customer_id with Customer.id also stored as a string, no coercion).

Correction (issue #1060 follow-up 2, 2026-08-22): the paragraph above, as first written, said the coerced form (toInteger(row.customer_id)) — the exact shape in the example at the top of this section — stayed a “known, separate, still-open limitation”. That is no longer true. A second #1060 follow-up taught the binding_seek mechanism (#429/#367) to seek through a CypherLit::Func value too, for a small, deliberately narrow allowlist of pure coercions: toInteger, toFloat, toString, toBoolean — exactly the four this manual already recommends for coercing a CSV cell (see “Cells are strings” above). So the foreign-key example at the top of this section, written exactly as shown, now seeks. The residual limitation is narrower than before, not gone: a coercion or transform OUTSIDE that four-function allowlist (toLower, abs, a +/arithmetic expression, …) still does not seed a seek, even where the function itself is pure — the allowlist is deliberately kept small (each addition is its own auditable decision, not “obviously fine, might as well”; see plan::expr::SEEK_SAFE_SCALAR_FUNCS’s doc comment for the reasoning) — and a value-dependent expression referencing a binding the match pattern itself introduces (not yet bound at seek time) still declines safely rather than seeking a wrong key. Gate: execution::in_seek_and_compound_range_tests:: unwind_seed_func_allowlist_each_coercion_seeks_and_matches_reference (all four coercions) and execution::load_csv_tests:: loadcsv_seed_func_tointeger_coercion_matches_unoptimised_reference (the real LOAD CSV path, this exact idiom).

CALL { … } IN TRANSACTIONS (below) bounds peak memory to one chunk regardless of whether a given write shape seeks — do not read chunking as a substitute for the seek when a shape falls outside the allowlist above.

Online + chunked: CALL { … } IN TRANSACTIONS

Wrapping the write subquery in CALL { … } IN TRANSACTIONS chunks the load into many small, independently-committed transactions — the online answer to the RAM cliff. Peak memory drops to one chunk, and on a cluster each chunk replicates as one small Raft entry instead of one unbounded one.

LOAD CSV WITH HEADERS FROM 'file:///data/more_people.csv' AS row
CALL {
  WITH row
  CREATE (:Reader {id: row.id, name: row.name, age: toInteger(row.age)})
} IN TRANSACTIONS OF 2 ROWS

Loading a 5-row file OF 2 ROWS produces ⌈5/2⌉ = 3 chunks — and three separate engrams, one per committed chunk:

MATCH (r:Reader) RETURN count(r) AS n
-- n: 5
CALL patinadb.engrams() YIELD id RETURN count(id) AS n
-- n: 3 (for this one load, on a fresh database)

• Each chunk is its own commit ⇒ its own engram, so batched ingest is versioned and time-travellable. A later chunk sees the writes committed by earlier chunks (read-your-writes).
• OF <n> ROW[S] sets the chunk size (default 1000).
• ON ERROR — FAIL (default) aborts the whole statement on a failing chunk; CONTINUE skips it and commits the rest; BREAK stops after it, keeping earlier chunks. A skipped/failed chunk changes nothing — it is resolved against a throwaway copy of the graph, so a partial chunk is never left behind.
• Crash-atomic per chunk: a crash between chunks recovers to a chunk boundary. The graph and the engram log always agree.
• The CALL { … } IN TRANSACTIONS must be the final clause of the query, and it cannot run inside an explicit Bolt BEGIN … COMMIT transaction.

On the server the driver proposes one client_write per chunk, so a 250-row load OF 100 ROWS advances the applied index by 3, not 1 — bounded Raft entries, read-your-writes preserved across chunks on the leader.

Server file-I/O sandbox

Reading a file:// URL, or writing with the export procedures, touches the server’s filesystem. On the server both are deny-by-default and gated by two layers:

1. Directory sandbox — --allow-csv-dir <dir> (reads) and --allow-export-dir <dir> (writes) whitelist directories; unset means every file access is refused. Paths are canonicalized, so .. traversal and symlink escapes are rejected.
2. Authorize by effect — any query that does file I/O is raised to require the global Admin role, because file access is a host-level capability, not a graph-data one. A per-database Writer cannot read or write host files.

/mgmt/upsert and its siblings confine their file path through the same --allow-csv-dir sandbox and are admin-only by virtue of living under /mgmt/. See Authentication & TLS → Cypher-driven file I/O.

Which path should I use?

• A large columnar file (or subgraph document) already on the server → POST /mgmt/upsert/upsert-edges/upsert-graph (fastest, index-accelerated match-or-create, chunkable).
• Live ingest into a running / clustered database from a streamed CSV → LOAD CSV … CALL { … } IN TRANSACTIONS OF n ROWS (versioned, replicated, bounded memory).
• A small load, or you need the CSV as a read source → plain LOAD CSV … CREATE / LOAD CSV … RETURN row.

Deferred index-build memory (#1398)

Index rebuilding externally merges all sorted runs before filling the index. It limits each spill run by both key count (PATINADB_INDEX_BUILD_SORT_KEYS, default 8,000,000) and retained key allocation bytes (PATINADB_INDEX_BUILD_SORT_BYTES, default 67,108,864 = 64 MiB). The byte accounting includes each key’s allocated capacity and vector descriptors and sequence bookkeeping. Both limits must admit a key into the current run. Zero or invalid values use the defaults. This applies to vertex/edge value, compound and edge-sorted rebuilds.

These are run-buffer limits, not a total process-memory limit. Allocator overhead, spare vector capacity, backend cache/transactions and other import buffers consume additional memory. A single key larger than the byte budget is flushed on its own; it is not rejected or silently omitted. The existing pending-build marker and terminal durability barrier remain unchanged. Size the total process/cgroup budget separately, including file cache, and compare finalization as well as insertion time.

Sorted-spool filling has a separate pair of limits: PATINADB_SORTMERGE_FILL_BATCH (default 200,000 entries) and PATINADB_SORTMERGE_FILL_BYTES (default 67,108,864 = 64 MiB of retained key/value allocations and descriptors). The fill stops a batch when either limit is reached; a single oversized entry is applied alone. Zero or invalid byte values use the default. This bounds the sorted consumer batch, not the producer’s external-sort spools (PATINADB_SORTMERGE_RUN_BYTES), the entire import, or all snapshot buffers. The shared sorted fill also serves snapshot installation; its surrounding atomic installation scope and crash-recovery markers are unchanged.

The offline importer no longer switches between inline and deferred loading. All imports use sorted mutation spools, including re-imports and skipped input reference checks. Existing stores apply bounded sorted chunks with incremental value, compound and edge-sorted index maintenance. Each chunk preserves properties absent from its input and removes superseded index keys before its terminal durability barrier; it does not rebuild unrelated indexes. Fresh whole-load builds retain their pending marker and derive indexes from final primary values when repeated input IDs prevent trusting precomputed value-index keys. The retired PATINADB_IMPORT_SORTMERGE and PATINADB_DEFERRED_INDEX_MIN_ROWS selectors no longer choose an alternative implementation. Index rebuild scratch files use private temporary directories under TMPDIR; size that disk for spilled keys.

Incremental delta transactions

Small input buffers do not necessarily imply small redb transactions. Updating scattered keys can modify many existing B-tree pages. When those dirty pages outgrow the write cache, early writeback can add I/O before the final durability barrier. Sorting the keys alone does not prevent this.

Existing-store sorted deltas therefore also use an estimated-work budget. Automatic sizing uses 1/32 of the configured redb cache, capped at 64 MiB, with a minimum charge of 16 KiB per mutation. This is a heuristic, not a bound on dirty pages, RSS or I/O, and does not promise faster imports for every workload. Fresh dense index construction and snapshot filling use their fill limits. Direct snapshot buffers also obey PATINADB_SORTMERGE_FILL_BYTES: a small record count can still retain large property values. This includes historical reconstruction for background tag snapshots, not just learner installation. Smaller delta chunks keep the same deferred-commit and terminal-sync contract; they do not make partially applied imports a supported recovery state.

PATINADB_SORTMERGE_DELTA_WORK_BYTES overrides this heuristic with a positive integer byte budget. Leave it unset initially. Reduce it when profiling shows write-cache churn during incremental import; raise it only when matched tests show that extra small transactions cost more than they save. The ordinary PATINADB_SORTMERGE_FILL_BYTES and PATINADB_SORTMERGE_FILL_BATCH caps remain independent upper limits. Raising the work budget does not bypass them.

For example, a configured 128-MiB redb cache selects a 4-MiB work budget. With the minimum charge, that admits at most 256 mutations per transaction; wider keys or values reduce the count. This is the policy’s arithmetic, not a measured optimal batch size. Set environment overrides before opening the database.

Compare the same database image, input and total RAM limit in both arms. Include terminal sync in wall time, check contents and indexes after reopen, and record CPU time, actual read/write bytes and I/O pressure. A lower byte count alone is not a latency improvement. Leave RAM for the OS page cache and other database buffers; enlarging redb’s cache inside an unchanged cgroup can create pressure elsewhere. Neither tuning knob replaces the process memory limit.

Bulk Upsert (match-or-create)

Development correction (#1320): the constraint checks and entity-complete graph-chunk boundaries described below are being qualified after0.21.0. Released0.21.0 bulk resolvers can bypass schema constraints; do not assume that a successful bulk request proves constraint validity on that revision.

Where Bulk Loading is about getting a lot of data in quickly, upsert is about reconciling data: for each input record, find an existing node (or edge) by a match key and UPDATE it, or CREATE it if it isn’t there. It is a bulk, index-accelerated MERGE — served online, over REST, by the leader.

By default, every upsert runs the versioned path — through the engram log, bracketed by a pre-load marker — so the whole load, however large, reverts as one unit (see Time Travel). With the default conflict policy, an ambiguous match rejects atomic resolution before graph changes. Chunked loads and lost responses have different recovery boundaries below.

There are three flavours, all admin- and leader-only REST endpoints:

All three accept a chunk_rows field: each chunk is one replicated Raft entry, so a very large load doesn’t ship as one giant entry (see Chunking a very large load). For /mgmt/upsert-graph, chunk_rows is a target number of resolved graph operations rather than input rows. A boundary extends to keep all changes to one entity together, including creation and its required properties. The entry can therefore exceed that target; the server’s entry-size limit still applies.

An ambiguous relationship endpoint fails explicitly. Node-file imports also offer the explicit on_conflict policies described below; none silently picks one arbitrary matching node.

Bulk resolution checks the final touched entities with the same uniqueness, required-property and property-type rules as Cypher writes, including secondary labels and relationship constraints. Validation uses disposable state, not partially applied live writes. Each committed chunk must also be valid on its own; rejecting a later chunk does not undo earlier committed chunks.

Node upsert — POST /mgmt/upsert

Match-or-create nodes on a per-batch match key. The file path is confined to the server’s filesystem via the same --allow-csv-dir sandbox LOAD CSV uses (see Bulk Loading → Server file-I/O sandbox).

curl -s -u neo4j:secret -X POST localhost:21001/mgmt/upsert \
  -H 'content-type: application/json' \
  -d '{
    "db": "default",
    "file": "/srv/import/people.csv",
    "label": "Person",
    "match": ["email"],
    "merge": "add",
    "on_conflict": "error",
    "strategy": "seek"
  }'

{"created": 2, "updated": 1, "skipped": 0, "conflicts": 0,
 "pre_load": "6f2f…", "post_load": "a1e3…",
 "pre_load_tag": "_upsert_pre_1699…",
 "revert_hint": "restore_to('6f2f…')"}

Per row, an existing :Person is found by exact equality on the match key (one attribute, or a composite ["first","last","dob"]). Found → UPDATE per merge; not found → CREATE. Re-running the same file is idempotent (it updates in place, never duplicates).

• merge: "add" (default) — add/overwrite the row’s attributes, keep the rest.
• merge: "replace" — the node keeps ONLY the row’s attributes (drops others).
• merge: "add-missing" — set only attributes the node does not already have.
• on_conflict: "error" (default) — a match key that hits more than one node fails loud. "update-all" updates every match; "skip" leaves them, counts them.
• strategy: "seek" (default) or "sort-merge" — two byte-identical lookup strategies; sort-merge reads the index in one sequential pass and is faster when the input is a large fraction of the label.
• chunk_rows — for a load too large to hold in RAM as one changeset (see below).

Concurrency safety. The resolve+propose runs through the same optimistic conflict-check pipeline REST autocommit writes use, so two concurrent upserts of the same value don’t both create a duplicate — the loser’s retry re-resolves against fresh HEAD and MATCHes the winner’s just-created node. This detection needs the match key to be a UNIQUE or NODE KEY constraint (see Constraints); an unconstrained match key is not conflict-detected online.

Chunking a very large load

By default an upsert is applied as one changeset (one engram), so the whole resolved batch is held in RAM. That is fine up to millions of rows, but a truly huge load (tens of millions of nodes/edges) can exhaust memory.

Pass chunk_rows to process the load in chunks of N rows, each applied as its own changeset — so peak memory stays bounded by the chunk size, not the total row count, and an arbitrarily large file loads:

curl -s -u neo4j:secret -X POST localhost:21001/mgmt/upsert \
  -H 'content-type: application/json' \
  -d '{"db":"default","file":"/srv/import/huge.parquet","label":"Person","match":["email"],"chunk_rows":100000}'

Rows whose keys an earlier chunk already created are matched, not duplicated. Every committed chunk must independently satisfy the schema constraints. Consequently, a whole-batch change such as exchanging two unique values may be valid atomically but invalid at an intermediate chunk boundary. The load remains revertable as one unit via its pre-load tag; chunking does not promise that every input accepted atomically will also succeed chunkwise.

Two things to know about the trade-off:

• Chunking is somewhat slower (each chunk pays a small per-changeset overhead) — a larger chunk_rows means less overhead but more memory.
• A chunked load is not all-or-nothing: if a chunk fails partway through (e.g. a non-unique match key), the earlier chunks stay applied. Recover using the pre-load HEAD/tag or reconcile the committed prefix. An error response need not contain the successful response’s revert hint. Without chunk_rows, a validation rejection before proposal leaves the graph untouched; use the default when the whole load must be atomic and fits in memory.

Constraint validation may briefly wait for a complete native read snapshot while another publication finishes. This wait consumes the request’s existing deadline; it does not extend the timeout, repeat committed chunks, or skip constraints. Embedded callers without a deadline receive the publication refusal immediately rather than waiting indefinitely.

A timeout or dropped connection is not evidence of rollback, even for an atomic request. The server may have committed the batch without its response reaching the caller. Persist the intended keys/values, check authoritative committed state with the appropriate causal barrier, and reconcile before retrying. Do not treat an initially absent write as proof that an in-flight request can no longer commit. Upserts do not automatically provide an unbounded exactly-once guarantee for arbitrary retry sequences.

/mgmt/upsert-edges takes the same chunk_rows field with the same semantics. /mgmt/upsert-graph applies all the subgraph’s nodes in chunks first (so an edge always finds its endpoints), then the edges in chunks. Note the JSON document itself is still held in memory as one coupled payload — chunking bounds the per-changeset memory, not the size of the document you can post in one call.

Edge upsert — POST /mgmt/upsert-edges

Match-or-create relationships between existing nodes. Each row’s start and end endpoints are resolved by the same match machinery as node upsert, then the edge is matched on its canonical (start, end, type) identity.

curl -s -u neo4j:secret -X POST localhost:21001/mgmt/upsert-edges \
  -H 'content-type: application/json' \
  -d '{
    "db": "default",
    "file": "/srv/import/works_at.csv",
    "start_label": "Person", "start_match": ["email"],
    "end_label": "Company", "end_match": ["name"],
    "rel": "WORKS_AT"
  }'

Every column named in start_match/end_match is consumed as an endpoint match value and is never an edge property; every other column is an edge property. Because patinaDB edges are id-less (at most one edge per (start,end,type)), there is no conflict policy for the edge itself — only on_missing governs a row whose endpoint match finds no node:

• "error" (default) — fail the whole load loud, touching nothing.
• "skip" — leave the row uncreated, count it, keep going.
• "create" — CREATE the missing endpoint node too, instead of failing or skipping. The created node’s properties are set to exactly its match-key attribute(s) — nothing else. Two rows in the same batch that reference the SAME missing endpoint (same label + same match value) resolve to the SAME newly-created node, so a file with many edges into one new node never creates duplicates.

{"created": 2, "updated": 0, "skipped": 0, "missing_endpoints": 1,
 "pre_load": "6f2f…", "post_load": "a1e3…",
 "pre_load_tag": "_upsert_edges_pre_1699…"}

Subgraph upsert — POST /mgmt/upsert-graph

The most powerful form: match-or-create a whole node+edge subgraph described in ONE coupled JSON document, posted as the request body, applied as one revertable engram.

Edges reference nodes by a fake id — a payload-local handle (unique within the document) used ONLY to wire edges[].from/.to to nodes[].id. A fake id is never stored; the server replaces each with the real DB UUID the node resolved to. This is what makes a homogeneous self-relationship (e.g. Person KNOWS Person) trivial — something a value-only edge match can’t express.

{
  "identity": { "Person": ["email"], "Company": ["name"] },
  "merge": "add",
  "nodes": [
    { "id": "p1", "label": "Person",  "props": { "email": "a@x", "name": "Alice", "age": 30 } },
    { "id": "c1", "label": "Company", "props": { "name": "Acme", "industry": "tech" } }
  ],
  "edges": [
    { "from": "p1", "to": "c1", "type": "WORKS_AT", "props": { "since": 2020 } }
  ]
}

curl -s -u neo4j:secret -X POST localhost:21001/mgmt/upsert-graph \
  -H 'content-type: application/json' --data-binary @graph.json

{"nodes_created": 2, "nodes_updated": 0, "edges_created": 1, "edges_updated": 0,
 "pre_load": "6f2f…", "post_load": "a1e3…",
 "pre_load_tag": "_upsert_graph_pre_1699…"}

Any number of node and relationship types

A subgraph document can mix as many node labels and relationship types as you like — each label is matched-or-created by its own identity, and the edges wire fake ids together across type boundaries. One document, one revertable engram:

{
  "nodes": [
    { "id": "p1", "label": "Person",  "props": { "email": "alice@x", "name": "Alice" } },
    { "id": "c1", "label": "Company", "props": { "name": "Acme" } },
    { "id": "s1", "label": "Skill",   "props": { "name": "Rust" } },
    { "id": "s2", "label": "Skill",   "props": { "name": "Graphs" } }
  ],
  "edges": [
    { "from": "p1", "to": "c1", "type": "WORKS_AT",  "props": { "since": 2020 } },
    { "from": "p1", "to": "s1", "type": "HAS_SKILL", "props": { "level": 5 } },
    { "from": "p1", "to": "s2", "type": "HAS_SKILL" }
  ]
}

Post this with an identity block naming Person=email, Company=name, and Skill=name — Alice is now connected to a Company and two Skills in one shot: three node types, two relationship types, resolved and applied together.

Where the match key (identity) comes from

Which properties make a node unique is resolved per label:

1. a persisted UNIQUE or NODE KEY constraint on the label — then you can omit the identity block entirely; else
2. the document’s identity block.

If both a constraint and a differing identity block exist, or neither exists, the load fails loud — patinaDB never guesses how to match your nodes.

An edge whose from/to names a fake id that isn’t in the document fails loud (the payload must be self-contained). merge (default "add") applies to node and edge properties, exactly as for node upsert.

The homogeneous case (Person KNOWS Person)

{
  "identity": { "Person": ["email"] },
  "nodes": [
    { "id": "p1", "label": "Person", "props": { "email": "a@x" } },
    { "id": "p2", "label": "Person", "props": { "email": "b@x" } },
    { "id": "p3", "label": "Person", "props": { "email": "c@x" } }
  ],
  "edges": [
    { "from": "p1", "to": "p2", "type": "KNOWS" },
    { "from": "p2", "to": "p3", "type": "KNOWS" }
  ]
}

The KNOWS edges land on exactly a→b and b→c (and NOT a→c) — the fake ids disambiguate which Person is which, even though every node has the same label.

Edge cardinality — 1-to-many vs 1-to-1

By default an edge is 1-to-many: many :WORKS_AT edges can leave one Person. Set an edge’s "identity" field to make it 1-to-1 — useful when a relationship is exclusive (one employer, one current owner, …).

1-to-many (default) — targets coexist. Load a-[:WORKS_AT]->Acme, then a-[:WORKS_AT]->Globex, and Alice works at both:

{ "from": "p1", "to": "c1", "type": "WORKS_AT" }

1-to-1 ("from") — the new target replaces the old. With a-[:WORKS_AT]->Acme already present, upserting:

{ "from": "p1", "to": "c2", "type": "WORKS_AT", "identity": "from" }

deletes the a→Acme edge and creates a→Globex — Alice now works at exactly one company. Re-upserting the same target just updates that edge’s properties.

Extra edge-property uniqueness — "match"

Because patinaDB edges are id-less, there is at most one physical edge per (from,to,type). An optional per-edge "match": ["attr",…] names edge properties that are part of the edge’s logical identity and acts as a guard: if a committed edge already occupies the triple but its match-property values differ from what you’re upserting, the load fails loud rather than silently overwriting a semantically-different edge. It composes with any cardinality mode.

Reverting a load

Every upsert is a single revertable changeset. The returned pre_load HEAD (and its replicated tag) is the exact state before the load. On the server:

USE default AS OF TAG '_upsert_pre_1699…'
MATCH (n) RETURN count(n)

Or inspect the whole load as one diff — CALL patinadb.diffRange('<pre_load>', '<post_load>') (see Diffs and Time Travel). The pre-load tag is replicated, so USE <db> AS OF TAG '<tag>' reads the graph as it was before the load on any node.

Treat pre_load_tag as an opaque returned identifier, not a timestamp to parse or construct. The #1323 fix (development, not yet released) derives new marker names from the exact pre-load Engram; older builds used seconds and could reuse a tag for different loads in the same second. Existing tags are not rewritten. On an older build, retain the explicit pre_load ID rather than trusting a shared generated tag to keep its earlier binding. The fix also covers offline CLI node, relationship and graph upsert markers. Marker prefixes remain intact; the suffix is not an API for extracting wall-clock time.

Rollback retention is an operational policy

Pre-load tags pin history snapshots. Frequent tiny management upserts therefore accumulate rollback artifacts; the snapshot cache setting is not a total budget for tagged history. Prefer ordinary Bolt writes for individual ticket updates and group nightly synchronization into a managed load (chunked when necessary). Retain each returned marker until the load is reconciled and its rollback window expires. Then an administrator can DROP TAG <returned_tag> if no other workflow still needs it; never blindly delete snapshot files. Dropping the last tag referencing a snapshot unpins it, but is not a promise of immediate disk reclamation. Monitor allocated disk and preserve restore/maintenance headroom.

The #1261 two-hour 4,096-ticket management-load drill retained 755 snapshot files occupying 1,163,825,152 allocated bytes on one source node, versus 10,862,592 allocated bytes for its live graph file. This is a fixture-specific warning, not an extrapolation to larger databases; see the backup qualification report.

Snapshot admission between graph chunks

Before proposing each resolved graph chunk, the server validates its constraints against a native snapshot. If a concurrent publication temporarily prevents a consistent snapshot, this read-only validation can wait within the load’s existing query deadline. Each attempt uses that same deadline; it is not reset for each chunk. With no configured deadline, snapshot refusal returns immediately. Constraint violations and write/commit errors are not retried by this handling.

This wait never repeats an already submitted chunk and does not make the whole load atomic. A terminal failure can still leave earlier chunks applied. It does not carry the non-chunked write_outcome: "not_submitted" receipt, and a client must not blindly replay the whole request. No new environment setting or timeout increase is introduced (#1483).

Constraints

patinaDB supports uniqueness, existence (NOT NULL), node-key, and property-type (IS :: <TYPE>) constraints on nodes, plus existence, property-type, uniqueness, and relationship-key constraints on relationships, with Neo4j-5 (and legacy Neo4j-4) DDL — all four Neo4j constraint kinds, on both nodes and relationships. Constraints are enforced at write time, replicate across a cluster, and are carried in backups and Raft snapshots.

Creating constraints

CREATE CONSTRAINT [name] [IF NOT EXISTS]
FOR (n:Label) REQUIRE n.prop IS UNIQUE

Node kinds:

Relationship kinds (FOR ()-[r:TYPE]-(), any direction):

Relationship uniqueness/key is index-backed by the existing edge-property value index (O(log N + matches)), not a full scan of the relationship type.

The legacy Neo4j-4 forms are also accepted: CREATE CONSTRAINT … ON (n:Label) ASSERT n.prop IS UNIQUE and the shorthand ON :Label(prop). The property-type predicate also accepts the IS TYPED <TYPE> synonym of IS :: <TYPE>.

Property types

IS :: <TYPE> (and IS TYPED <TYPE>) accepts: BOOLEAN, STRING, INTEGER, FLOAT, POINT, DATE, LOCAL TIME, ZONED TIME, LOCAL DATETIME, ZONED DATETIME, DURATION, and LIST (BOOL/INT are accepted as aliases). A missing or null property is allowed — a type constraint restricts a present value’s type only; combine it with an existence constraint if you also need the property present. Element-type refinement (LIST<INTEGER NOT NULL>) is not yet supported (bare LIST only).

CREATE CONSTRAINT person_email FOR (p:Person) REQUIRE p.email IS UNIQUE
-- status: 'Constraint created on :Person(email) IS UNIQUE'

CREATE CONSTRAINT emp_key FOR (e:Employee) REQUIRE (e.dept, e.num) IS NODE KEY
-- status: 'Constraint created on :Employee(dept, num) IS NODE KEY'

The optional name identifies the constraint for DROP and SHOW. IF NOT EXISTS makes creation idempotent.

Creating over existing data

CREATE CONSTRAINT first validates the current graph. If the existing data already violates the constraint (a duplicate value, or a missing required property), the statement fails and no constraint is registered — fix the data and re-run.

Enforcement semantics

Enforcement runs at write time on CREATE, MERGE, and SET / REMOVE — including a SET / REMOVE inside a FOREACH body, which is enforced identically to a top-level write:

CREATE (:Person {email: 'ada@x.io', name: 'Ada'})

CREATE (:Person {email: 'ada@x.io', name: 'Ada2'})
-- error: Unique constraint violation on :Person(email): value already exists

• UNIQUE is NULL-exempt. A null (or absent) value is not constrained — many nodes (or relationships) may omit prop. Only two concrete equal values collide. (Use a node-key/relationship-key or a separate existence constraint if you also need the property present.)
• 1 and 1.0 are the same value. Uniqueness treats an integer and a mathematically-equal float as identical (matching Neo4j), so under UNIQUE(k) a stored Integer 1 collides with a CREATE/SET of 1.0, and vice-versa. This holds for node UNIQUE, relationship UNIQUE, and every column of a composite NODE KEY / REL KEY, on both the single-writer path and concurrent transactions. The fold is exact — a float merges with an integer only when it is finite, whole, and round-trips bit-for-bit — so two genuinely different values never collide, and NULL stays exempt as above. See Cypher → numeric equivalence for the full rule. (Very large exact-integer float values — roughly ≥ 2^59, e.g. 1e18 — are still enforced correctly, but fall back to a scan instead of an index seek, so those checks can be slower.)
• Existence and node-key/relationship-key reject a missing/null required property — including a CREATE/MERGE that omits it, a SET n.prop = null, or a REMOVE n.prop. This holds for both node existence and relationship existence (checked at edge create, SET r.prop = null, SET r =/+=, and REMOVE r.prop).
• Property-type rejects a present wrong-typed value on CREATE/MERGE/SET (node or relationship). A missing/null value is allowed.
• The uniqueness check is an O(log N + matches) seek of the label-scoped value index (node) or the edge-property value index (relationship), so it is cheap even on a large label/type.
• Node-key uniqueness is served by an automatically registered backing compound index over the key tuple — you do not create it separately. Relationship-key uniqueness is served the same way over the edge-property value index (no separate index to create).

Inspecting and dropping

Schema introspection (SHOW CONSTRAINTS, SHOW INDEXES, SHOW POINT INDEXES, SHOW EDGE SORTED INDEXES and SHOW FUNCTIONS) is read-only, including when query memory admission is enabled. Creating or dropping schema objects still uses the replicated write path in server mode. A client timeout does not prove that a schema change was rolled back: inspect the resulting catalog before deciding how to proceed.

SHOW CONSTRAINTS

constraints: ['person_email: :Person(email) IS UNIQUE', 'emp_key: :Employee(dept, num) IS NODE KEY', 'person_name: :Person(name) IS NOT NULL']

Relationship constraints look the same, scoped by type instead of label, e.g. REQUIRE (r.a, r.b) IS REL KEY FOR ()-[r:WORKS_AT]-().

DROP CONSTRAINT person_email [IF EXISTS]

Replication and durability

• Cluster replication is automatic. Constraint DDL replicates through the Raft log and re-runs deterministically on every node (the same IndexDdl path that index DDL uses).
• Raft snapshots carry the defs. A follower that bootstraps purely from a streamed snapshot (after the DDL log is purged) still enforces every constraint.
• Portable backups carry the defs too. GET /mgmt/snapshot / POST /mgmt/restore round-trip UNIQUE, existence, and node-key defs (alongside vector indexes, tags, and RBAC users) — restore re-issues them as CREATE CONSTRAINT … IF NOT EXISTS. See the REST API.

Not supported

• LIST<T> element-type refinement — only bare LIST is accepted.
• Bare composite UNIQUE without existence — use IS NODE KEY / IS [REL|RELATIONSHIP] KEY for a multi-property key (which also requires presence).

See Limitations for the full schema-feature scope, and the Data Model for how properties and the value index work.

Edge-Sorted Indexes

An edge-sorted index keeps, per anchor vertex, that anchor’s neighbours over one relationship type pre-ordered by a property of the neighbour. Eligible ordered pages use this index to select candidates instead of starting with an unordered traversal. Residual filters, additional hops and secondary ordering still require work; an API page size is not a fixed bound on candidates read.

This is a patinaDB extension. The grammar reference lives in Cypher Support → DDL; this chapter is the conceptual and worked-example companion.

The problem it solves

One shape shows up nearly everywhere you build on a graph:

• a user’s most recent tickets,
• the newest comments under a post,
• a person’s activity feed / timeline,
• the top-N items per parent, ordered by a timestamp, score, or price.

In Cypher it reads:

MATCH (u:User {name: 'Ada'})-[:REPORTED]->(t:Ticket)
RETURN t
ORDER BY t.created_at DESC
LIMIT 10

Simple to write, but normally expensive. To return ten rows the engine must:

1. traverse the whole fan-out — every REPORTED edge Ada has, even if she filed 50 000 tickets,
2. read the sort property (created_at) off every one of those targets, then
3. sort all of them just to discard everything below the top ten.

The work is proportional to the anchor’s degree, not to the LIMIT. For a hub vertex — a power user, a popular post, a busy parent — that is the difference between a feed that renders instantly and one that stalls.

What patinaDB does

An edge-sorted index stores a sorted adjacency: for each anchor, its targets over one relationship type are already laid out in target-property order. The query above becomes:

1. seek to the anchor’s slice of the index,
2. iterate in created_at order (DESC walks it in reverse),
3. take k and stop.

That is O(n + k) (skip n, take k) — no fan-out, no materialising the whole neighbour set, no post-sort. A power user with 50 000 tickets costs the same top-ten as one with twelve. The index is maintained synchronously on every write (edge create/delete, and moving a target when its sort property changes), so it is never stale.

The DDL and a worked example

The syntax names the traversal shape and the target property to order by. Both orientations are supported:

-- outbound anchor: (anchor)-[:REL]->(target)
CREATE EDGE SORTED INDEX [name] FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at

-- inbound anchor: (target)<-[:REL]-(anchor)
CREATE EDGE SORTED INDEX [name] FOR (m:Ticket)<-[:HAS_LABEL]-()  ON m.created_at

SHOW EDGE SORTED INDEXES

Fast-path eligibility is narrower than valid Cypher syntax. Besides the basic single-key shape, RFC 0035 adds direct-property secondary ordering, deterministic residual filters and fixed-hop continuation for a single resolved anchor. Identity WITH target projections can retain this access path. Computed sort expressions and renamed sort bindings are not covered by this extension. Keep a stable tie-breaker where your pagination contract requires one; do not remove it merely to obtain an index hit. Verify the entire query with EXPLAIN, including filters, cursor and WITH clauses.

The name is optional and cosmetic — a def is identified by its shape (direction, relationship type, target label, target property), so two CREATEs of the same shape are idempotent.

The following was run end-to-end against a live server on a tiny User → REPORTED → Ticket graph; the output is real.

Build the graph

CREATE (u:User {name:'Ada'})
CREATE (t1:Ticket {id:1, title:'Login fails',    created_at:'2026-07-01T09:00:00'})
CREATE (t2:Ticket {id:2, title:'Slow dashboard', created_at:'2026-07-03T14:30:00'})
CREATE (t3:Ticket {id:3, title:'Export broken',  created_at:'2026-07-05T08:15:00'})
CREATE (t4:Ticket {id:4, title:'Typo in header', created_at:'2026-07-06T11:45:00'})
CREATE (u)-[:REPORTED]->(t1)
CREATE (u)-[:REPORTED]->(t2)
CREATE (u)-[:REPORTED]->(t3)
CREATE (u)-[:REPORTED]->(t4)

Create the index

$ patinadb feeddb query \
    "CREATE EDGE SORTED INDEX ticket_feed FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at"
status: 'Edge sorted index ticket_feed created FOR ()-[:REPORTED]->(:Ticket) ON created_at'

$ patinadb feeddb query "SHOW EDGE SORTED INDEXES"
edgeSortedIndexes: ['FOR ()-[:REPORTED]->(:Ticket) ON created_at']

Creating the index over a populated graph backfills the existing edges, so you can create it any time.

Page 1 — newest first

$ patinadb feeddb query \
    "MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
     RETURN t.id, t.title, t.created_at
     ORDER BY t.created_at DESC LIMIT 2"
t.created_at: ['2026-07-06T11:45:00', '2026-07-05T08:15:00']
t.id: [4, 3]
t.title: ['Typo in header', 'Export broken']

Confirm the index serves it

EXPLAIN shows the physical plan. When an edge-sorted index covers the query, the Physical: footer names limit.edge_sorted_topk:

$ patinadb feeddb query \
    "EXPLAIN MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
     RETURN t ORDER BY t.created_at DESC LIMIT 2"
plan: 'Limit skip=0 count=2
  Project items=1 distinct=false return_star=false
    Sort keys=1
      MatchScan required=[(u:User) -[entry](t:Ticket) → PropertyIndex[created_at] SORTED SCAN DESC est.5]
Physical:
  limit.edge_sorted_topk
'

If you drop the index (or the query shape does not match), the footer names a different path — always run EXPLAIN to confirm the index is doing the work.

Page 2 — keyset pagination

To page forward, carry a cursor: the sort value of the last row on the previous page (2026-07-05T08:15:00), and ask for rows strictly beyond it. For a DESC feed that is <:

$ patinadb feeddb query \
    "MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
     WHERE t.created_at < '2026-07-05T08:15:00'
     RETURN t.id, t.title, t.created_at
     ORDER BY t.created_at DESC LIMIT 2"
t.created_at: ['2026-07-03T14:30:00', '2026-07-01T09:00:00']
t.id: [2, 1]
t.title: ['Slow dashboard', 'Login fails']

This single-value example assumes distinct creation times. For duplicate timestamps, use the stable two-key cursor below; a strict timestamp-only cursor would skip remaining rows with the boundary timestamp.

Stable cursors, filters and additional hops

Use a unique tie-breaker and carry both ordered values from the last row:

This recipe assumes non-null cursor fields and uniqueness across result rows, not merely target vertices. If a join produces several rows for one target, its identifier alone cannot distinguish those rows at a page boundary. Include the required continuation identity in the application’s ordering/cursor, or deliberately deduplicate before pagination (which may select the fallback plan). Nullable cursor fields need explicit null-aware predicates; ordinary =, < and > comparisons to null do not match null rows.

MATCH (author:Author {author_id:$author})
      <-[:AUTHORED_BY]-(comment:Comment)<-[:HAS_COMMENT]-(ticket:Ticket)
WHERE split(ticket.issue_key, '-')[0] IN $allowed
  AND (comment.created_at < $after_time
       OR (comment.created_at = $after_time AND comment.comment_id > $after_id))
WITH comment
ORDER BY comment.created_at DESC, comment.comment_id ASC
LIMIT $limit
RETURN comment.comment_id, comment.created_at, comment.body

The corresponding existing-format index is:

CREATE EDGE SORTED INDEX
FOR (comment:Comment)<-[:AUTHORED_BY]-() ON comment.created_at

The first page omits the cursor predicate. When the complete shape is eligible, EXPLAIN reports limit.edge_sorted_topk. The reader seeks using a proven necessary primary-key bound, keeps the boundary group, and still evaluates the original cursor and permission predicates. Secondary ordering is resolved inside complete primary groups. Only matching rows count toward SKIP/LIMIT; multiple matching ticket continuations retain their Cypher row multiplicity.

The extension currently admits a uniform primary value domain, across the indexed target label/property, of strings, integers, booleans or legacy timestamps, with missing properties handled by ordinary null ordering. Mixed primary domains and unsupported types fall back. Temporal Cypher values are distinct from legacy timestamps; this extension does not add multikey temporal acceleration. Optional source matches, variable-length source paths, DISTINCT before the page, unsupported expressions and insufficient secondary-label coverage also retain ordinary execution. Optional hydration after the page is separate and can be used without changing the inner page’s eligibility.

Large primary tie groups and restrictive permission filters can still require many reads. A secondary key is not physically stored in this single-property index, and no universal constant-time or fixed-read-count guarantee is made. The index and residual traversal use the same query snapshot, but independent page requests do not share a frozen snapshot: concurrent changes to ordering keys can move rows across pages. Use a supported fixed historical view when the application needs a frozen feed.

The RFC 0035 extension describes the development build, not a released-version guarantee. Component tests cover the unchanged Jira-shaped request, planner selection, duplicate continuations and pinned snapshots. A three-hour mixed regional simulation passed on the frozen development build (#1261, 2026-09-21); first-touch aggregation and long-tail filtered author pages remain performance limitations. This is not cold-start, real-WAN or production approval, nor a fixed latency guarantee per page.

Pages across an anchor frontier

A read-only query can build a frontier through earlier hops or UNWIND, then page over a final directed, fixed-length relationship to indexed targets:

MATCH (root:Account {id:$id})-[:FOLLOWS*1..2]->(author:Account)
WITH collect(DISTINCT author) AS authors
UNWIND authors AS author
MATCH (author)<-[:AUTHORED_BY]-(post:Post)
WHERE post.created_at < $before
RETURN post.id AS id, post.created_at AS created
ORDER BY created DESC, id ASC
SKIP $skip LIMIT $limit

Declare CREATE EDGE SORTED INDEX FOR (p:Post)<-[:AUTHORED_BY]-() ON p.created_at to make the final-hop index available. A regular property range index does not supply these per-anchor streams. Eligible plans report limit.edge_sorted_topk. The executor merges the streams, retains duplicate frontier rows and resolves complete primary ties before stopping. Only matching rows consume the page.

This frontier path currently supports uniform integer, boolean or legacy timestamp primary values, including missing values. Strings, floats, mixed primary types, secondary-only target label coverage and unsupported final-hop shapes retain ordinary execution. Earlier traversal still runs normally; a large frontier, large ties or restrictive residual filters can require substantial work. Index creation and maintenance have their ordinary costs. Independent page requests still observe independent snapshots.

When to use it — and when not

Reach for it when you have the “top-N neighbours by a neighbour property” shape over a skewed fan-out (some anchors have far more neighbours than others) and you want the first page to stay fast regardless of degree. Feeds, timelines, “latest N per parent”, and leaderboards-per-group are the sweet spot.

Each index covers exactly one (direction, relationship type, target label, target property) combination. Different primary sort properties or relationship types need separate definitions. Supported secondary sort keys do not require another index, but their tie-group work remains at read time.

Create it after a bulk load. Bulk import does not maintain edge-sorted indexes incrementally; CREATE EDGE SORTED INDEX after the load backfills the whole graph in one pass, and every write from then on keeps it live.

Skip it when the fan-out is small and uniform (a plain traverse-then-sort is already cheap), when you never LIMIT the result (you want all neighbours in order — there is no top-N to accelerate), or when the target property changes very frequently on high-degree anchors (see the write-cost note below).

Advisor: discovering you need one

You do not have to guess whether a query would benefit. patinaDB ships an advisor that recognises the traversal-fan-out + ORDER BY target.prop LIMIT shape and, when it is running the slow fan-out top-K only because no covering edge-sorted index exists, tells you the exact statement to create. It is advice only — it never creates anything and never changes how a query runs.

In EXPLAIN / PROFILE. When the plan falls onto the fan-out limit.bounded_topk for a coverable shape, the plan text carries an Advice: line naming the index that would flip it to limit.edge_sorted_topk:

EXPLAIN MATCH (u:User {uid: 1})-[:REPORTED]->(t:Ticket)
        RETURN t ORDER BY t.created_at DESC LIMIT 20

…
Physical:
  limit.bounded_topk
Advice: CREATE EDGE SORTED INDEX FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at  (would serve this ORDER BY … LIMIT via edge_sorted_topk instead of a fan-out top-K)

Run that CREATE EDGE SORTED INDEX and the advice disappears — the plan now reads limit.edge_sorted_topk. That round-trip is the guarantee: an advice is emitted iff creating the named index would actually engage the fast path (it reuses the executor’s own coverage decision), so there are no false positives.

As a procedure. CALL patinadb.advisor(query) analyses an arbitrary query string and returns one row per suggestion:

CALL patinadb.advisor(
  'MATCH (u:User {uid: 1})-[:REPORTED]->(t:Ticket) RETURN t ORDER BY t.created_at DESC LIMIT 20'
) YIELD suggestion, reason, current_plan

It returns no rows (no error) when there is nothing to suggest: a non-traversal query, a shape an edge-sorted index cannot serve (variable-length, undirected, multi-anchor, no ORDER BY … LIMIT, a filtered/unlabeled target), or a query already served by a covering index. Use it to sweep your hot read queries and discover the exact indexes to declare.

Honest positioning and tradeoffs

How this compares to other graph databases. General-purpose graph databases, Neo4j included, index a relationship’s own properties, or a node’s properties — not a per-anchor adjacency pre-sorted by a neighbour’s property. For the top-N-neighbours-by-neighbour-property shape that means the usual plan is traverse-then-sort, exactly the cost this index removes. The underlying idea of a sorted adjacency is not new — bespoke feed and social-graph stores have long kept time-sorted association lists for precisely this access pattern — but exposing it as a declarative, general-purpose index you can CREATE over any (rel, target-property) pair, inside a Cypher database, is uncommon. That is the differentiator, stated plainly.

Write cost. The index is maintained synchronously, and the cost is not free:

• Creating or deleting an edge updates one index entry — cheap.
• Changing a target’s sort property moves that target in every anchor’s slice that points at it. For a target with high in-degree (many anchors point to it), a single SET t.created_at = … is O(in-degree) index moves. If your workload rewrites the sort property often on well-connected targets, weigh that against the read speedup.

Snapshot-carry caveat (clustered mode). In a Raft cluster the index definitions replicate via the Raft log — every node re-runs the DDL and builds its own copy, deterministically. They are not yet carried in Raft snapshots. A node that bootstraps purely from a streamed snapshot (after the log that carried the CREATE was purged) will lack the def and quietly fall back to traverse-then-sort until the DDL is re-issued. Correctness is never affected — only speed — and the fallback path returns identical rows. Re-issue the CREATE EDGE SORTED INDEX statements after such a bootstrap (they are idempotent) to restore the fast path. This is tracked in Limitations.

How the Query Planner Works

This chapter is the structured reference for the patinaDB query planner. It is written for application developers and operators who write Cypher and read EXPLAIN and PROFILE. It explains which stage makes each decision, what inputs that stage uses, and which settings change the outcome. With it you should be able to predict the plan for a query and explain it.

Query Planning & Performance keeps the detailed, feature-by-feature notes. This chapter links to it where that chapter has more detail.

Two rules hold throughout:

• Plans never change results. Every alternative described here returns the same rows. Row order is the same wherever ORDER BY defines it. Without ORDER BY, row order is unspecified and can differ between plans and between cluster nodes. Statistics only choose plans; they never filter rows, and they are never replicated. Different access paths can visit different records, so which storage error a damaged record raises, and when, can depend on the plan.
• The RFC 0042 cost model and operators are on by default. Since 2026-10-17 the cost model (join), factorized execution and the auto modes of worst-case optimal joins and the CSR adjacency cache are the shipped defaults (#1442). The reference planner, with heuristic rules over cardinality estimates, remains: it decides wherever the cost model declines or keeps it by margin, and setting all four switches to off restores it exactly (see Specialised operators and switches).

The pipeline at a glance

A read query passes through these stages, in this order:

Read this table as two halves. Everything up to the plan cache is decided once per query text and then reused. Everything in the execution row is decided again on every run, using the current statistics.

Figure 1. The pipeline. The plan cache stores the rewritten operator tree; the entry choice (with the A2 entry-decision cache, introduced in RFC 0042 A6), fast paths and operator admission are decided on every execution.

The plan cache

The plan cache holds the operator tree after the access-path rewrite. That tree fixes the join order and the per-step hash or nested-loop choice.

• Scope. One cache per process holds up to 512 plans, keyed per database, and evicts the least recently used. There is no setting for its size. PATINADB_PLAN_CACHE=off (also 0, false or no) disables it, together with literal normalization.

• Key. A plan is reused only for the same normalized text. User parameters ($personId) are kept by name, so every value of $personId shares one plan.

• Literals. Literals are replaced by internal parameters. Normalization is skipped, and the plan is keyed by the exact text, when:

  • the query has two or more MATCH steps that reach join reordering;
  • the text contains comments, backticks or non-ASCII characters;
  • the normalized text does not parse.

• Decided per execution, not cached.

  • the entry node and the access path of each MatchScan;
  • the order of a multi-pattern MatchScan;
  • all fast paths, limit rules and aggregate specialisations;
  • parallel admission and the auto modes.

• Invalidation. Cached plans are dropped by:

  • any DDL, such as CREATE INDEX or CREATE EDGE SORTED INDEX;
  • clearing the graph (restore, snapshot install, reset);
  • DROP DATABASE.

  Under PATINADB_COST_MODEL=join, an accepted degree-catalog publication for the same database also makes the cached plan rebuild on its next lookup. Refused refreshes, ordinary data writes and publications in another database do not trigger this rebuild. Reference/entry plans retain their previous policy. Parameter values still share a plan; parameter-sensitive join planning remains a limitation between replans.

• EXPLAIN and PROFILE plan afresh. They never read the plan cache. If the data or the statistics have changed since a query was first cached, EXPLAIN can therefore show a different join order from the one the cached plan executes. Under PATINADB_COST_MODEL=join, the decisions show a plan.cache … chosen=built line, which records the schema epoch the join decisions were made under.

The L3 result cache, which stores whole result tables, is separate; see Cache memory budget.

The plan.cache wording in the example below reflects this publication policy; the original example fixture and the other recorded decisions are unchanged.

Entry node and direction

Each MatchScan starts at one node of its path, the entry node, and expands from there in both directions. This choice is made again on every execution.

Figure 2. Entry choice. The margin depends on the provenance of the cheapest candidate’s inputs; the details follow in the two sections below.

Reference rules (fallback)

The reference selector decides wherever the entry cost model declines a pattern or keeps its pick by margin, and everywhere when PATINADB_COST_MODEL=off. It applies these rules in order:

1. UUID shortcut. A node constrained by an id equality (id(n) = $id, elementId(n) = …) is the entry, without estimating anything else.
2. Single node. A pattern with one node starts there.
3. Sort hint. If ORDER BY names a property of a node in the pattern, that node is the entry. The exception is when its estimate is at least 10 times the cheapest node’s estimate (integer ratio, both estimates positive).
4. Minimum cardinality. Otherwise the labelled node with the smallest estimate wins.
   • Ties go to the leftmost node.
   • The walk stops at the first estimate of 1 or less.
   • Unlabelled nodes are never estimated.
5. Direction check. For chains of 3 to 9 nodes the planner can then switch to the other endpoint. This applies only when every node has exactly one label and no properties, and every relationship has a fixed length and one type. If the two endpoint estimates are within 2× of each other, it samples up to 32 anchors per hop and direction, with one shared 16,384-edge budget. There is no setting for this step.

The node estimates come from these rules (constants as in the source; they are heuristics, not measured optima):

The first label is the primary label. Label counts, value indexes and compound indexes are keyed by it. A label that a node holds only as a secondary label (for example Post in CREATE (:Message:Post)) is not counted by the label index. The reference selector then falls back to the total vertex count. In the example dataset this is why (:Comment) is estimated at est rows=31070, the whole graph, instead of 20,000.

Once the entry node is chosen, the access path for it is the first rule that applies, in this priority order:

Priority is fixed with one exception. When both a compound index and a single-property index apply, they are compared on an estimated rows + io cost. If the cost beats the priority order, the Physical: footer says so with an entry.cost_override: … line (see Cost-based selection).

The entry cost model (PATINADB_COST_MODEL=entry)

With PATINADB_COST_MODEL=entry (or join, the default, which includes it), the minimum cardinality rule (step 4) and the direction check (step 5) are replaced, for eligible patterns, by a whole-path cost in calibrated nanoseconds.

• Eligible patterns. Fixed-length paths with at least one relationship, where every relationship has exactly one type. Patterns with an id constraint, variable-length relationships, shortestPath, untyped or type-alternative relationships, and patterns with a sort hint keep the reference rules. So do patterns whose estimates are not finite, and patterns whose planning scratch memory is refused.

• What is priced. Every labelled node competes as the entry. Each candidate costs:

  • its entry access (seek or scan);
  • for every entry row, the expansion of both sides of the path, including label checks and property reads at each hop;
  • the construction of the cross product of both sides.

  The hop sizes come from the degree catalog, using the mean degree for the (relationship type, direction, anchor label, other label) group. Filters are assumed independent.

• Regimes. The primitive costs were measured on one host (A2-cal v1, #1442). A hop is priced as “warm L2” only when the entry cache is enabled, its expected list fits the L2 admission cap (at most 8,192 edges), and the working set fits the level cap. Otherwise it is priced as “warm redb”. The cold regime is never chosen automatically.

• Keyed consumers. Some operators repeat work per run of one binding. Today this is only the count over independent OPTIONAL one-hop branches. For those, the expected number of runs times the per-run cost is added as entry.cost.consumer.

• Adoption margin. The model’s choice replaces the reference entry only when the predicted advantage clears a margin that depends on the inputs:

  • Exact inputs: no margin; the cheapest candidate is used.
  • Sampled degrees: the larger of 1.1× and the spread of the candidates’ estimated output rows.
  • Missing or stale degrees (Default, Stale): 2×.

  Ties keep the reference entry. These margins are unverified policy choices, not confidence intervals.

• Missing statistics. If a degree group is missing or stale, the model uses a cached legacy fan-out or a default of 1. It also asks for one bounded background refresh, which runs on a single worker:

  • budget PATINADB_DEGREE_STATS_EDGE_BUDGET, default 1,000,000 directional edges; 0 disables the refresh;
  • a 120-second deadline;
  • a 5-second retry backoff per database.

  The query that triggered the refresh does not wait for it. The refresh= field of entry.cost reports what happened, for example started, not needed, single flight or cache budget refusal. A refusal means the entry cache is disabled, or the stats.degree region has less than the 256-group minimum left. A 2 GiB node fits the sample (#1442). So the default cost model does not need a manual CALL patinadb.stats.degrees(): the first queries of a shape plan with the fallback, later ones with the sample once it is published. The sample is node-local RAM, lost on restart, and costs a bounded edge sample plus one pass over all vertices on a background thread. Join orders chosen before it arrived stay in the plan cache until a schema change or eviction (see The plan cache); entry choices are re-priced when the catalog changes.

• It needs the entry cache. The degree catalog lives in a region of the entry cache. A server enables the entry cache by default (PATINADB_CACHE_LIMIT=40%). Embedded use enables it only when PATINADB_CACHE_LIMIT is set. Without it, every degree lookup falls back to Default and the refresh is refused. This is the same IC2-shaped query run embedded without a cache limit:

  entry.cost _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"]: chosen=node[0] (est rows=1.931123270035404) (est ns=4155.373289255539) over node[2] (est rows=1.931123270035404) (est ns=844442.8018863855) over node[4] (est rows=60) (est ns=25337715.354519267); minimum calibrated primitive ns; dominant=entry access; warm redb default, L2 only within list and working-set caps; A1 population mean or legacy-default cache-only fallback; refresh=cache budget refusal; independent filters; constants A2-cal v1 #1442; Default; repeats=3

All values and their side effects are listed in Entry and join cost model.

Joins

Which patterns become a join

Before any ordering, lowering folds patterns that share an endpoint into a single path. For example, (a)-[:R]->(b), (b)-[:S]->(c) becomes one path (a)-[:R]->(b)-[:S]->(c). Patterns stay separate when:

• a pattern is OPTIONAL or binds a named path;
• both patterns contain a relationship and come from different MATCH clauses;
• the shared variable is an interior node of either path (branches and stars stay separate);
• two patterns share both endpoints but use disjoint relationship types. They stay a two-key join instead of becoming one closed walk (LSQB Q2’s shape; see Closed paths and endpoint joins).

Separate MATCH clauses lower together like comma patterns. The exceptions are an OPTIONAL clause, a clause with its own WHERE or dynamic pattern properties, and a clause with two or more relationship patterns. Patterns that remain separate form a join chain.

Reference join planner (fallback)

The reference join planner orders joins when join costing declines or is switched off.

1. Seed. The pattern with the smallest base estimate starts the chain; ties go to the first pattern. The base estimate is the smallest node estimate in the pattern. A pattern that touches a variable bound by an enclosing seed counts as 1.

2. Order. Up to 8 patterns use a Selinger-style dynamic program that minimises the sum of intermediate result sizes. Above 8 patterns, a greedy order is used.

   • A connected join step is estimated as max(prefix, base), a foreign-key assumption. It does not use fan-out.
   • A disconnected step is estimated as prefix × base.
   • Connected patterns always come before disconnected ones. The planner never adds a cartesian product that the query did not already force.
   • Finally, patterns that are anchored on two or more bound nodes, or that close a cycle on a bound node, are moved forward.

3. Strategy per step. A step becomes a seeded nested loop (Apply) when it is connected and either:

   • anchored as above; or
   • a point probe applies (one hop, or every interior node bound) and the nested-loop cost prefix × seek beats a scan of the right side.

   The right side must have at least 1,000 estimated rows, except when the whole step is evaluated per outer row. Every other step is a hash join (Join kind=hash); the right side is the build side.

4. Cartesian products. A keyless join fails with a resource error once it would emit more than PATINADB_CARTESIAN_CAP rows (default 10,000).

5. Semi-join pruning. In supported shapes, the hash table of a join is passed as a membership filter into a later traversal. That traversal then skips vertices that cannot join (up to 8 keys).

6. Partial COUNT. An ungrouped count(*) over a keyed hash-join chain is rewritten into per-side grouped counts. These are joined on the keys and summed as products. In EXPLAIN this shows as a Join of two Aggregate group_by=… inputs, as in Example C.

Hash and cartesian joins emit combined rows on demand (#1468), preserving left scan order and right bucket/scan order, including duplicates. Under an active query memory context, new row backing and recursive scalar copies are admitted before allocation against the existing query/transaction budgets. A memory refusal terminates output with ResourceExhausted. The cartesian cardinality gate still checks the complete product before emitting any row. Probe key containers, recursive scalar copies and edge-formatting scratch also reserve before allocation (#1469). Their credit stays with the current probe and releases on advancing, exhaustion, error or early iterator drop, rather than accumulating in delivery. Missing bindings retain their skip semantics. Keyed build tables also admit their hash allocation, fresh composite keys and bucket storage (#1470), including old/new capacity overlap during growth. Shared membership filters retain the same table owner; the last owner releases its credit. Existing input row payloads are moved or shared, not re-accounted as new table payload. Join input collectors reserve slot capacity before growing (#1471), including old/new allocation overlap. Cartesian iterators keep both input permits until drop; keyed build releases its collector after handoff. Collection can fail its memory budget before the product cardinality is known. No new switch is introduced. Shared seed/import selection and APPLY/OPTIONAL merges now use the same fallible copier (#1473). Unique owned entries move; shared entries require an admitted recursive copy. CALL import visibility and first-binding precedence remain unchanged. Refusals propagate through reader, writer, predicate and COUNT/EXISTS callers. A governed APPLY iterator checks its original query memory context on every advance, including within the current seed. Copied payload credit stays with query delivery, including discarded temporary copies. Original row producers and other copy sites remain separate limitations, so this is not a complete join memory bound. The keyed emitter probes its immutable bucket per output row; its throughput impact has not been measured.

OPTIONAL MATCH never joins the chain. It becomes a LeftJoin that seeds each optional pattern from every left row. Under a query memory context, its input, clause extensions, results and memoized candidate row-slot buffers reserve capacity before growth/copy (#1472). Sort input and correlated/rooted MATCH candidate and accumulator containers use the same owner. Concurrently live containers count together; completed output keeps its credit through iterator/result lifetime. A refusal propagates through the existing resource-error path. Identity/SKIP filtering, clause-level WHERE and NULL extension, duplicate order and count/deadline guards are unchanged. The existing limits therefore may reject work previously admitted with uncounted buffers. Optional memo keys/table storage and recursive row/model producers remain separate limitations. No new knob or measured throughput claim.

Cost-based join ordering (PATINADB_COST_MODEL=join)

join adds degree-aware join costing on top of entry:

• Search.

  • Up to 10 relations: a Pareto (cost, rows) subset dynamic program. It gives up and falls through when a frontier would need more than 8 states.
  • Up to 32 relations: an interval dynamic program over the greedy order.
  • Above 32: greedy.

  These limits are unverified policy. The search is left-deep and adds no cartesian products.

• Pricing. For each step, a hash join (build + probe + row construction) is compared with a seeded nested loop (seeks plus expansion per outer row), in the same calibrated nanoseconds as the entry model. No new operator is introduced.

• Guard. The reference plan is priced in the same model. The search result is adopted only if it is predicted to be faster by more than 1.1× (when both prices come from exact statistics) or 2× (otherwise). The reference plan is also kept when:

  • every pattern is edge-free;
  • a pattern closes a cycle, meaning a node variable repeats inside one pattern. A cycle spread over several comma patterns does not count;
  • an input cannot be modelled: shortest paths, variable-length or multi-type relationships;
  • the reference price is not finite, or planning scratch memory is refused;
  • the reference executes as a partial COUNT and the search result would not. The audit then says kept reference: it executes as a partial COUNT ….

On the LSQB, SF1 and Velr suites that were checked (#1442), the guard keeps the reference plan everywhere (RFC 0042, Increments A3). Treat join as an experiment that reports its reasoning, not as a tuning knob.

Statistics

Notes:

• Generations. A generation advances once per dirty label or relationship type at each storage synchronisation, not once per written row.
• Where statistics live. Statistics are kept in RAM only. The cache holds at most 4,096 entries or 4 MiB, is cleared completely when it overflows, and is lost on restart. Each cluster node computes its own statistics; they are never replicated.
• Estimates vs guarantees. Every number here is an estimate. Statistics are never used to prove that a row is absent or to skip a check that results depend on.
• CALL patinadb.stats(label?) shows exact, freshly computed values, including secondary-label holders. It is not a view of the planner’s cached (possibly stale, primary-label-only) entries, so its numbers can differ from the estimates in EXPLAIN.

The degree catalog

CALL patinadb.stats.degrees(edgeBudget?) builds the RFC 0042 A1 degree catalog and returns one row per (relationship type, direction, anchor label, other label) group. Both labels include secondary labels. Here is an excerpt from the example dataset:

edge_type | direction | anchor_label | other_label | edges | anchors | population | p50 | p99 | max | heavy_hitters | provenance | sample_edges | sample_anchors
  …
HAS_CREATOR | in | Person | Message | 30000 | 1000 | 1000 | 15 | 110 | 110 | [16 entries] | exact | 30000 | 1000
  …
KNOWS | in | Person | Person | 6500 | 1000 | 1000 | 7 | 9 | 9 | [16 entries] | exact | 6500 | 1000
KNOWS | out | Person | Person | 6500 | 1000 | 1000 | 7 | 7 | 155 | [16 entries] | exact | 6500 | 1000

How to read a row:

• anchors counts anchors with at least one edge, while population includes zero-degree anchors.
• p50 and p99 are upper bounds of log₂ buckets.
• heavy_hitters lists up to 16 high-degree candidates with their exact observed degrees. The cost model uses an exact heavy-hitter degree only for a literal UUID seed; for anything else it uses the mean.
• Exact vs sampled. Without an argument, the budget is PATINADB_FANOUT_RECOMP_MAX_EDGES. If both directions of the graph fit the budget, the pass is exact. Otherwise it samples 256 evenly spaced UUID ranges, assuming uniformly distributed UUIDs. sampled counts are then extrapolated, max is only the observed maximum, and groups that the sample missed are absent rather than proven empty.
• Population. The population counts come from a full vertex scan, which the edge budget does not bound.
• Failure. An exhausted budget, more than 4,096 groups, or a deadline fails the call and keeps the previous catalog.
• Freshness. A group counts as stale for the entry and join models as soon as any of its three generations (relationship type, anchor label, other label) has moved. Factorized admission uses a looser rule: it refreshes when the generations have moved by more than max(64, population / 100).
• Storage. The catalog is stored only when the entry cache is enabled (see above). Planner paths never block on building it; they request the background sample described in The entry cost model.

Specialised operators and switches

All five are read once per process; restart after changing them. --check-config validates their values and prints the effective mode of the four RFC 0042 switches, marking defaults. Note that PATINADB_WCOJ rejects Boolean aliases: at runtime true and 1 mean off.

Defaults (since 2026-10-17, #1442). Unset or empty means the default in the table. An explicitly set value keeps the meaning it had before the defaults changed: off selects the reference path, and so does any value outside a switch’s grammar (--check-config rejects those). To run the reference planner and executor exactly as before, set all four to off:

PATINADB_COST_MODEL=off PATINADB_FACTORIZE=off PATINADB_WCOJ=off PATINADB_CSR=off

The default was chosen from the server comparison in Measured effects: Velr 0.544, LSQB 0.878, SF1 0.995 of all-off. PATINADB_QUERY_THREADS stays 1.

What auto means. Under auto, an operator engages only where it has its own cost admission, and only when PATINADB_COST_MODEL is entry or join. Without a cost model, auto behaves exactly like off. Operators without a cost admission (anti-intersection, the cyclic join, CSR shortest paths and CSR fixed hops) need an explicit on.

Figure 3. Switch map. “Uncosted” means the operator engages wherever its structural checks pass; “inert” means an auto mode without a cost model.

Prerequisites that decline silently. Factorized execution, sorted intersections and the cyclic join run only under a query memory scope:

• On a server, Bolt reads get one from the memory governor.
• Embedded code gets one from Dataset::query_table_read_with_memory.

Without a scope, PROFILE reports agg.factorized aggregate: chosen=flat over factorized; query memory scope required and the reference operator runs.

Factorized execution (PATINADB_FACTORIZE)

Factorized operators work on a fixed path that splits at an interior pivot into two independent branches. Instead of building every combination of the two branches, they read each branch once per pivot.

• Grouped aggregates (B2).
  • Aggregates: count(*), count(x) and count(DISTINCT …), with group keys on the pivot or on the branch end nodes.
  • Exact-integer sum, min and max on a general path, but only with the cost model off.
  • Branches may share a relationship type only if they reach the pivot through it in the same way. Relationship uniqueness is then checked per pivot at run time.
• B3–B5. DISTINCT over the pivot plus one branch, top-k within a factor, and cross-branch =/<> counts by inclusion–exclusion. A degenerate factor flattens back to the ordinary operator, before any output. The details are in Factor consumers.
• Admission.
  • With the cost model off, every proven shape engages (forced; meant for tests and measurement).
  • With the cost model on, only the per-pivot count path is cost-admitted. It is admitted when 700 · F > 1.5 · (1000 + 300 · W + 30 · F) ns, where F is the estimated flat rows and W the walked elements per pivot, taken from the degree catalog. The weights were fitted on one host and are indicative (#1442). Missing or refresh-due degree evidence keeps the flat aggregate. SUM/MIN/MAX and join shapes are not costed and stay flat.
  • B3–B5 have no cost admission. They engage whenever the switch is on and their structural checks pass.

The same Velr-A1-shaped query (“pairs of authors whose comments reply to the same post”), first with PATINADB_FACTORIZE=on alone and then with the cost model and auto modes on as well:

  agg.factorized pivot=4 (per-pivot key multisets; runtime witness proof for shared relationship types; runtime admission)
  …
  agg.factorized aggregate: chosen=factorized over flat; selected; cached pivot count=Some(0); cached first-hop means=[Some(2.0), Some(2.0)]; PATINADB_COST_MODEL records its estimate separately; Cached; repeats=1

  agg.factorized pivot=4 (per-pivot key multisets; runtime witness proof for shared relationship types; runtime admission)
  …
  agg.factorized aggregate: chosen=flat (est rows=4) (est cost=2800) over factorized (est rows=4) (est cost=2320); cost model per pivot: flat rows=4.00, walk elements=4.00, flat=2800ns, factorized=2320ns, margin=1.5; Cached; repeats=1

Both runs return the same five rows. On this graph each post has two comments, so F = 4 flat rows per pivot against W = 4 walked elements. That fails the margin, and the cost model keeps the flat aggregate. The grouped count MATCH (p:Person)-[:KNOWS]->(q:Person)<-[:HAS_CREATOR]-(m:Message) RETURN q.lastName AS name, count(*) AS n ORDER BY name LIMIT 3 pivots at q. It has F = 195 against W = 36.5 and is admitted:

  agg.factorized pivot=2 (per-pivot key multisets; runtime admission)
  …
  agg.factorized aggregate: chosen=factorized (est rows=195) (est cost=17800) over flat (est rows=195) (est cost=136500); cost model per pivot: flat rows=195.00, walk elements=36.50, flat=136500ns, factorized=17800ns, margin=1.5; Cached; repeats=2

Worst-case optimal joins (PATINADB_WCOJ)

• C0, closing probe (expand.into). When the last hop of a pattern returns to an already bound node, the executor can probe that one edge instead of scanning the whole adjacency and filtering. The closing relationship must be unnamed, single-typed and without properties. Under auto the probe is chosen when the cached list length or the catalog degree makes a full scan dearer than one seek; unknown degrees keep the scan.
• C1, sorted intersection (expand.intersect). For c in N(a) ∩ N(b) shapes (triangles, mutual friends), the executor merges two UUID-sorted neighbour lists instead of expanding and filtering. Under auto, each candidate samples at most 256 keys per side. The intersection is used only when the binary plan’s intermediate result exceeds the output and the binary plan costs more than 1.5× the intersection. Sampling stops for a hop after 32 candidates whose expected gains do not pay for it, and resumes on every 64th. The anti-intersection for NOT (a)--(c) runs only with on.
• C2, cyclic join (join.wcoj). Connected cyclic regions with at least three fixed, typed relationships (up to 16 vertex variables and 16 relationships) run as a leapfrog join over the adjacency. Named or property-filtered relationships join it as binary probes. OPTIONAL, variable-length paths, derived inputs and scans with ordering promises keep the reference execution. Runs only with on; there is no cost admission yet.
• Triangle entry (entry.intersect). With auto and a cost model, a homogeneous undirected triangle may be entered at the node that gives C1 its complete prefix.

Limitations: Experimental cyclic joins require explicit activation.

CSR adjacency cache (PATINADB_CSR)

The CSR is a derived, read-only, in-memory copy of one relationship type’s adjacency in one direction. It is built in the background from the stored trees and is never written to disk or replicated.

• When it is used. Until a CSR is built, and for any anchor written since the build, the existing reader is used. A CSR more than a quarter dirty is rebuilt.
• Memory. Its share is one ninth of the decoded entry-cache levels’ allowance; the four existing levels keep two ninths each. With on, that share is reserved at start-up. With auto, it is reserved only once the CSR cache exists.
• auto. Only deduplicating variable-length traversals (BFS), where count(DISTINCT …) or similar makes path multiplicity irrelevant, are cost-admitted. The expected expansions must save more than twice the dense-id lookups plus a 1/4096 share of the build scan; this policy is unverified. Shortest paths keep the reference reader.
• on. Also enables shortest paths and single-type fixed hops (D2).

EXPLAIN neither builds nor reports a CSR. Only PROFILE shows traversal.source:

  csr.admission deduplicated BFS: chosen=csr candidate; degree/visits versus amortized build and lookup; uncertified caches still decline; Structural; repeats=1
  traversal.source "KNOWS" reverse=false: chosen=csr over l2/redb; clean anchor at compatible snapshot; Exact; repeats=225
  traversal.source "KNOWS" reverse=true: chosen=csr over l2/redb; clean anchor at compatible snapshot; Exact; repeats=225

In the same auto run, the C1 intersection’s sorted domains also consulted the CSR and requested its build (not built, then background build in progress in Example D). That run had no csr.admission decision for the triangle query. That excerpt was captured before the CSR auto containment: in the current source, auto admits only the deduplicating BFS, and intersection domains stay on the reference reader without requesting a build. They read the CSR only with PATINADB_CSR=on.

Parallel execution (PATINADB_QUERY_THREADS)

With a thread cap above 1, eligible read-only scans, fixed traversals, mergeable aggregates (not sum/avg), bounded top-k, sorted DISTINCT and seeded Apply can run as morsels on the query pool. Results and their order keep the serial contract. Admission depends on the cost model:

• Cost model off. A scan or traversal goes parallel only above PATINADB_PAR_AGG_MIN_ROWS estimated rows (default 100,000, unverified).
• Cost model on (A4). A plan goes parallel only when its estimated serial work exceeds 4 × (400 µs dispatch + 265 ns per morsel + 1.4 ns per row), and the parallel version is also cheaper. Some cases need more:
  • aggregates grouped by an entity add about 1 µs of serial work per group;
  • row-producing pipes still also need the row floor;
  • a serial specialisation (path count, covering input) keeps priority unless the parallel plan is four times cheaper.
• Always.
  • Workers must be idle (parallel.load in the access profile).
  • Deferred-prefix pruning always keeps priority.
  • Edge morsels split a first hop only when its actual average fan-out reaches PATINADB_PAR_FANOUT_MIN_EDGES (default 50,000).

A two-hop grouped count (MATCH (a:Person)-[:KNOWS]-(b:Person)-[:KNOWS]-(c:Person) RETURN c.lastName AS name, count(*) AS n) with PATINADB_QUERY_THREADS=4. With the cost model off, the row floor compares the entry estimate (1,000) with 100,000 and stays serial:

      [2] MatchScan required=[(a:Person) -(b:Person) -(c:Person)] rows=377064 est=1000 q-error=377.064
  …
  parallel.admission traversal rows: chosen=serial over parallel (est rows=1000); max label estimate >= floor 100000; entry=1000; Cached; repeats=2

With PATINADB_COST_MODEL=join, the calibrated work estimate is compared with the overhead and the pool admits four threads:

      [2] MatchScan required=[(a:Person) -(b:Person) -(c:Person)] rows=377064 est=169000 q-error=2.231
  …
  parallel.admission calibrated work: chosen=parallel (est rows=169000) (est cost=2830130.3734979997) over serial (est rows=169000) (est cost=8780502.616999999); serial_ns > 4 * overhead_ns=635004.719248; effective_threads=4; morsels=4; A2-cal v1, unverified margin; Cached; repeats=2
  …
  parallel.admission pool load: chosen=parallel over serial; idle=8/8; requested cap=4; admitted cap=4; Exact; repeats=1
  …
  parallel.load: admitted, idle 8/8, threads 4

More detail is in Day-2 operations and the PATINADB_QUERY_* rows of Configuration.

Fast paths

These run automatically, without a switch:

• Limit rules. Rules for ORDER BY … LIMIT are tried in this priority order, and the walk is recorded as limit.rule:
  1. edge_sorted_topk (needs an edge-sorted index);
  2. spatial_knn;
  3. reverse_probe, which is cost-compared against the fan-out top-k;
  4. bounded_topk, an O(k) heap;
  5. window.
• Path counts. agg.disjoint_path_count and agg.grouped_disjoint_path_count count fixed paths that split into branches with disjoint relationship types, without building cross-product rows.
• Optional counts. agg.independent_optional_count handles count(*) over up to 8 independent optional one-hop branches.
• Covering input. agg.covering (PATINADB_READ_COVERING, default on) applies to broad integer aggregates of at least 4,096 estimated rows.
• Whole-query counts. Label, filtered and grouped count scans.

The full list is in Fast paths that make queries cheap.

Reading Decisions: in EXPLAIN and PROFILE

EXPLAIN returns a plan string. In order, it contains:

• the operator tree, read bottom-up;
• Storage: and property-layout lines;
• an optional Physical: footer with planned rules and cost lines;
• Decisions:;
• optional Advice: and Warning [...] lines.

PROFILE runs the query and also returns elapsed_ms. Its plan string adds:

• Planned access (before execution): and the same planned tree;
• Executed operators (query thread): with actual row counts;
• Decisions (executed):, the decisions taken during this execution;
• the executed aggregate access, the resolved property storage and the storage request counts.

Each decision line has this form:

  <point> <subject>: chosen=<alternative> [(est rows=…)] [(est ns|cost=…)] over <alternative> …; <reason>; <provenance>; repeats=<n>

• point: the decision family. Examples are entry.node, entry.access, entry.cost, join.order, limit.rule, agg.factorized, parallel.admission and traversal.source.
• subject: what was decided. This is usually the pattern, with labels and directions, or a step such as step 1.
• chosen / over: the chosen alternative first, then the rejected ones. Priority walks stop at their winner, so later alternatives may be missing.
• estimates: est ns is used for entry.cost and for decisions whose reason starts with calibrated ns; other estimates are in the selector’s own units (est cost). They are estimates, not timings.
• reason: the rule or the margin that decided.
• provenance: where the inputs came from.
  • Exact: a fresh, exact statistic.
  • Sampled { size }: sampled evidence.
  • Cached: a value reused from a cache or catalog.
  • Histogram: column histogram interpolation.
  • Stale: an outdated statistic.
  • Default: no statistic; a fallback value was used.
  • Structural: a rule with no numeric estimate.
• repeats: how often the same point, subject and choice recurred.
  • Seeded and nested executions repeat decisions.
  • The estimates shown are from the first occurrence.
  • At most 128 distinct decisions are kept.

In Executed operators, [n] is the preorder index of the executed plan, and rows= counts rows actually pulled from that operator. A LIMIT can leave its input only partly consumed. rows=unobserved (bypassed/fused) marks an operator that a fast path replaced or fused. A MatchScan (and, under PATINADB_COST_MODEL=join, a join) also shows its estimate and its q-error: the larger of estimate/actual and actual/estimate, with zero counted as one. What the estimate means depends on the mode:

• Cost model off. The MatchScan estimate is the entry node’s estimate, not the path’s output. A multi-hop scan therefore shows a large q-error even when the plan is right.
• Cost model on. The estimate is the chosen entry’s predicted output rows from entry.cost, which is directly comparable to rows=.

A large q-error shows where an estimate was wrong. It does not mean the plan was wrong. Worker-internal rows are not shown.

The examples below are real output from an in-process release build of this source on the generated example dataset (1,000 persons, 30,000 messages; #1442). Lines marked … are omitted. The query texts are the SF1 Interactive and LSQB texts from crates/patinadb/src/plan/exec/ldbc_pins/ (IC2, IC9) and ldbc_pins/parallel_admission.rs (LSQB). The row counts are logical work on a toy graph and are no performance measurement.

Example A: an IC2-style friends’ messages page

MATCH (:Person {id: $personId})-[:KNOWS]-(friend:Person)<-[:HAS_CREATOR]-(message:Message)
WHERE message.creationDate <= $maxDate
RETURN friend.id AS personId, friend.firstName AS personFirstName, friend.lastName AS personLastName,
       message.id AS postOrCommentId, coalesce(message.content, message.imageFile) AS postOrCommentContent,
       message.creationDate AS postOrCommentCreationDate
ORDER BY postOrCommentCreationDate DESC, toInteger(postOrCommentId) ASC
LIMIT 20

EXPLAIN with every switch off (personId = 1):

Limit skip=0 count=20
  Sort keys=2
    Project items=6 distinct=false return_star=false
      MatchScan required=[[entry](_:Person) -(friend:Person) -(message:Message) → PropertyIndex[id] = est.1]
Storage: native (allocated database fits detected RAM at open)
Vertex property storage: owner-key directory; canonical values <= 256 serialized bytes inline, wider values in property/owner payload table; automatic physical routing, existing query access operators retained
Physical:
  limit.bounded_topk
  parallel.topk: serial: query thread cap is one
  parallel.frontier: serial: query thread cap is one
Decisions:
  entry.node _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"]: chosen=node[0] (est rows=1); minimum cardinality; stop at <=1; unlabelled nodes skipped; Default; repeats=3
  entry.direction admission: chosen=fallback; shape, cardinality separation or sampling unavailable; retain pivot; Structural; repeats=3
  entry.access None: chosen=property_value_index; static priority; no complete cost comparison; Structural; repeats=3
  limit.rule edge_sorted_topk: chosen=declined; priority walk; stop at first applicable rule; Structural; repeats=2
  limit.rule spatial_knn: chosen=declined; priority walk; stop at first applicable rule; Structural; repeats=2
  limit.rule reverse_probe: chosen=declined; priority walk; stop at first applicable rule; Structural; repeats=2
  limit.rule bounded_topk: chosen=applicable; priority walk; stop at first applicable rule; Structural; repeats=2

• entry.node … chosen=node[0] (est rows=1): the id index equality gives an estimate of 1. The walk stops there, so the other nodes are never estimated.
• entry.access … property_value_index: the entry is a point lookup on the value index (PropertyIndex[id] = est.1 in the tree).
• limit.rule: no edge-sorted index covers this sort, so the walk ends at bounded_topk, a 20-row heap instead of a full sort.

With PATINADB_COST_MODEL=join and the degree catalog loaded, the same EXPLAIN prices all three entries:

Decisions:
  entry.cost.hops _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"] node[0]: chosen=estimate (est rows=376.5690376569038) (est cost=38508.96192405824); right[0] scanned=13.000 output=13.000 check_ns=8.933533 record_probes=0 regime=warm L2; right[1] scanned=390.000 output=376.569 check_ns=8.933533 record_probes=0 regime=warm L2; ; Exact; repeats=3
  entry.cost.hops _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"] node[2]: chosen=estimate (est rows=376.56903765690384) (est cost=3045511.749845188); left[0] scanned=13000.000 output=13.000 check_ns=8.933533 record_probes=0 regime=warm L2; right[0] scanned=30000.000 output=28966.849 check_ns=8.933533 record_probes=0 regime=warm L2; ; Exact; repeats=3
  entry.cost.hops _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"] node[4]: chosen=estimate (est rows=376.5690376569038) (est cost=40711683.6658395); left[0] scanned=28966.849 output=28966.849 check_ns=8.933533 record_probes=0 regime=warm L2; left[1] scanned=376569.038 output=376.569 check_ns=8.933533 record_probes=0 regime=warm L2; ; Exact; repeats=3
  entry.cost _:["Person"] - ["KNOWS"]:Undirected - friend:["Person"] - ["HAS_CREATOR"]:Inbound - message:["Message"]: chosen=node[0] (est rows=376.5690376569038) (est ns=38508.96192405824) over node[2] (est rows=376.56903765690384) (est ns=3045511.749845188) over node[4] (est rows=376.5690376569038) (est ns=40711683.6658395); minimum calibrated primitive ns; dominant=expansion; warm redb default, L2 only within list and working-set caps; A1 population mean or legacy-default cache-only fallback; refresh=not needed; independent filters; constants A2-cal v1 #1442; Exact; repeats=3
  entry.access None: chosen=property_value_index; static priority; no complete cost comparison; Structural; repeats=3

• entry.cost.hops shows, for each candidate, the scanned and accepted rows per hop, the label-check cost and the regime (warm L2). For node[0], 13 KNOWS neighbours are expected (the mean undirected degree), then 390 messages, 376.6 of which pass the date filter.
• The cost model picks the same entry as the reference. Every input is Exact, so no margin applies.

PROFILE then shows how far the estimate was off:

Executed operators (query thread):
  [0] Limit skip=0 count=20 rows=20
    [1] Sort keys=2 rows=unobserved (bypassed/fused)
      [2] Project items=6 distinct=false return_star=false rows=unobserved (bypassed/fused)
        [3] MatchScan required=[(:Person) -(friend:Person) -(message:Message)] rows=5300 est=376.5690376569038 q-error=14.074

Person 1 is one of ten hubs in the dataset, with 155 outgoing KNOWS edges. The catalog’s mean degree cannot see that, hence the q-error of 14. Only a literal UUID seed would use the exact heavy-hitter degree. The plan is still the best one available: a point entry followed by expansion.

Example B: IC9 and an edge-sorted index

IC9 collects friends and friends of friends, then pages their messages by date. Without a covering index, PROFILE with every switch off shows the seeded Apply expanding HAS_CREATOR once per friend:

Executed operators (query thread):
  [0] Retain keep=[personId,personFirstName,personLastName,commentOrPostId,commentOrPostContent,commentOrPostCreationDate] rows=20
    [1] Limit skip=0 count=20 rows=20
      [2] Sort keys=2 rows=unobserved (bypassed/fused)
        [3] Project items=7 distinct=false return_star=false rows=unobserved (bypassed/fused)
          [4] Apply imports=[] scope=InheritOuter rows=17550
            [5] Retain keep=[friend,message] rows=565
              [6] Unwind binding=friend rows=unobserved (bypassed/fused)
                [7] Aggregate group_by=0 aggs=1 rows=1
                  [8] Filter rows=2344
                    [9] MatchScan required=[(root:Person) -(friend:Person)] rows=2344 est=1 q-error=2344.000
            [10] MatchScan required=[(friend:_) -(message:Message)] rows=unobserved (bypassed/fused)
  …
  traversal.source fixed ["HAS_CREATOR"] reverse=true: chosen=redb over csr over l2; CSR off or auto (fixed hops require on); Exact; repeats=565
  …
Resolved vertex property storage (query thread; logical resolutions, not physical I/O; workers/caches/buffers may bypass):
  inline=42100 canonical bytes=241541; external=0 canonical bytes=0

After CREATE EDGE SORTED INDEX FOR (m:Message)<-[:HAS_CREATOR]-() ON m.creationDate (run on a copy of the dataset), the limit.edge_sorted_topk rule applies. The ordered frontier then merges the friends’ date-sorted edge lists. The HAS_CREATOR traversal reader is entered 20 times instead of 565 (see repeats):

Physical:
  limit.edge_sorted_topk
  …
Executed operators (query thread):
  [0] Retain keep=[personId,personFirstName,personLastName,commentOrPostId,commentOrPostContent,commentOrPostCreationDate] rows=20
    [1] Limit skip=0 count=20 rows=20
      [2] Sort keys=2 rows=unobserved (bypassed/fused)
        [3] Project items=7 distinct=false return_star=false rows=unobserved (bypassed/fused)
          [4] Apply imports=[] scope=InheritOuter rows=unobserved (bypassed/fused)
            [5] Retain keep=[friend,message] rows=565
              [6] Unwind binding=friend rows=unobserved (bypassed/fused)
                [7] Aggregate group_by=0 aggs=1 rows=1
                  [8] Filter rows=2344
                    [9] MatchScan required=[(root:Person) -(friend:Person)] rows=2344 est=1 q-error=2344.000
            [10] MatchScan required=[(friend:_) -(message:Message)] rows=unobserved (bypassed/fused)
  …
  traversal.source fixed ["HAS_CREATOR"] reverse=false: chosen=redb over csr over l2; CSR off or auto (fixed hops require on); Exact; repeats=20
  …
Resolved vertex property storage (query thread; logical resolutions, not physical I/O; workers/caches/buffers may bypass):
  inline=92 canonical bytes=506; external=0 canonical bytes=0

The Apply is now fused (unobserved), and 92 property resolutions replace 42,100. These are logical counts from one run each, not timings. The q-error of 2,344 on [9] compares the entry estimate (1, the id lookup) with the 2,344 rows that the KNOWS*1..2 expansion produced. With the cost model off, that is expected. The cost model would not change it either, because it does not price variable-length paths.

Example C: an LSQB-style count

LSQB Q2 with PATINADB_COST_MODEL=join:

MATCH (person1:Person)-[:KNOWS]-(person2:Person),
  (person1)<-[:HAS_CREATOR]-(comment:Comment)-[:REPLY_OF]->(post:Post)-[:HAS_CREATOR]->(person2)
RETURN count(*) AS count

Planned access (before execution):
Project items=1 distinct=false return_star=false
  Aggregate group_by=0 aggs=1
    Join kind=hash keys=[person1,person2]
      left:
        Aggregate group_by=2 aggs=1
          MatchScan required=[[entry](person1:Person) -(person2:Person) → LabelScan ⚠ (no matching index) est.1000]
      right:
        Aggregate group_by=2 aggs=1
          MatchScan required=[(person1:Person) -(comment:Comment) -(post:Post) -[entry](person2:Person) → LabelScan ⚠ (no matching index) est.1000]
  …
Executed operators (query thread):
  [0] Project items=1 distinct=false return_star=false rows=1
    [1] Aggregate group_by=0 aggs=1 rows=1
      [2] Join kind=hash keys=[person1,person2] rows=269
        [3] Aggregate group_by=2 aggs=1 rows=13000
          [4] MatchScan required=[[entry](person1:Person) -(person2:Person) → LabelScan ⚠ (no matching index) est.1000] rows=13000 est=13000 q-error=1.000
        [5] Aggregate group_by=2 aggs=1 rows=20000
          [6] MatchScan required=[(person1:Person) -(comment:Comment) -(post:Post) -[entry](person2:Person) → LabelScan ⚠ (no matching index) est.1000] rows=20000 est=20000 q-error=1.000
Decisions (executed):
  join.strategy step 1: chosen=Hash (est rows=31070) over NestedLoop (est rows=1000); connected and anchored/probe eligible; otherwise hash; Structural; repeats=1
  join.order bases=[1000.0, 31070.0]: chosen=[0, 1] (est rows=31070) (est cost=32070); DP (cap 8), anchored priority; Default; repeats=1
  …
  join.order person1:["Person"] - ["KNOWS"]:Undirected - person2:["Person"] | person1:[] - ["HAS_CREATOR"]:Inbound - comment:["Comment"] - ["REPLY_OF"]:Outbound - post:["Post"] - ["HAS_CREATOR"]:Outbound - person2:[]; relations=2; guard: chosen=reference [0, 1] (est rows=260) (est ns=34999058.497517996) over subset DP [1, 0] (est rows=403.90999999999997) (est ns=33889480.401324295); calibrated ns; kept reference: predicted advantage below 1.1x; Exact; repeats=1
  plan.cache join planning epoch: chosen=built; schema_epoch=0; retained decisions; data writes retain plans; new degree catalogs replan; Cached; repeats=1

How to read it:

• The join. The two patterns share both endpoints but use disjoint relationship types, so they stay separate join inputs.
• The rewrite. The partial COUNT rewrite turned the hash join into a join of two grouped counts: 13,000 and 20,000 grouped rows meet in 269 key matches.
• The reference decisions. join.order (DP over 2 patterns) and join.strategy (hash) were made by the reference planner. bases=[1000.0, 31070.0] shows the secondary-label fallback for the second pattern.
• The guard. The join.order … guard line is the A3 search. It found a different order 3% cheaper, which is below the 1.1× margin for exact inputs, so the reference order was kept.
• The cache entry. plan.cache … chosen=built records the schema epoch of these decisions.
• Estimate quality. Both MatchScan estimates have a q-error of 1.000. Under the cost model they are the predicted output rows from the degree catalog. With the switches off, the same scans show the label-count entry estimate of 1,000.

A keyed consumer looks like this. These lines are from LSQB Q7 under PATINADB_COST_MODEL=entry:

  entry.cost.consumer _:["Tag"] - ["HAS_TAG"]:Inbound - message:["Message"] - ["HAS_CREATOR"]:Outbound - creator:["Person"] node[0]: chosen=keyed runs (est cost=18684342.197546996); message:LIKES runs=30000.000 ns_per_run=305.783; message:REPLY_OF runs=30000.000 ns_per_run=317.028; ; Exact; repeats=1
  …
  entry.cost.consumer _:["Tag"] - ["HAS_TAG"]:Inbound - message:["Message"] - ["HAS_CREATOR"]:Outbound - creator:["Person"] node[2]: chosen=keyed runs (est cost=18684342.197546996); message:LIKES runs=30000.000 ns_per_run=305.783; message:REPLY_OF runs=30000.000 ns_per_run=317.028; ; Exact; repeats=1
  …
  entry.cost.consumer _:["Tag"] - ["HAS_TAG"]:Inbound - message:["Message"] - ["HAS_CREATOR"]:Outbound - creator:["Person"] node[4]: chosen=keyed runs (est cost=18684342.197546996); message:LIKES runs=30000.000 ns_per_run=305.783; message:REPLY_OF runs=30000.000 ns_per_run=317.028; ; Exact; repeats=1
  …
  entry.cost _:["Tag"] - ["HAS_TAG"]:Inbound - message:["Message"] - ["HAS_CREATOR"]:Outbound - creator:["Person"]: chosen=node[0] (est rows=30000) (est ns=29063112.205906995) over node[2] (est rows=30000) (est ns=38630507.417546995) over node[4] (est rows=30000) (est ns=29369677.606546998); minimum calibrated primitive ns; dominant=keyed consumer; warm redb default, L2 only within list and working-set caps; A1 population mean or legacy-default cache-only fallback; refresh=not needed; independent filters; keyed-consumer runs priced; constants A2-cal v1 #1442; Exact; repeats=1

Example D: a triangle count under WCOJ

MATCH (a:Person)-[:KNOWS]-(b:Person)-[:KNOWS]-(c:Person)-[:KNOWS]-(a)
RETURN count(*) AS triangles

With COST_MODEL=join WCOJ=auto CSR=auto FACTORIZE=on, under a query memory scope:

Executed operators (query thread):
  [0] Project items=1 distinct=false return_star=false rows=1
    [1] Aggregate group_by=0 aggs=1 rows=1
      [2] MatchScan required=[(a:Person) -(b:Person) -(c:Person) -(a:_)] rows=26760 est=338000 q-error=12.631
  …
  entry.cost a:["Person"] - ["KNOWS"]:Undirected - b:["Person"] - ["KNOWS"]:Undirected - c:["Person"] - ["KNOWS"]:Undirected - a:[]: chosen=node[2] (est rows=338000) (est ns=22099296.471) over node[0] (est rows=338000) (est ns=82355575.995) over node[4] (est rows=338000) (est ns=23492927.619); minimum calibrated primitive ns; dominant=row construction; warm redb default, L2 only within list and working-set caps; A1 population mean or legacy-default cache-only fallback; refresh=not needed; independent filters; constants A2-cal v1 #1442; Default; repeats=1
  entry.intersect homogeneous triangle: chosen=node[0]; calibrated closing-domain candidate beats binary expansion; complete prefix required; Structural; repeats=1
  …
  expand.intersect bounded closing domains: chosen=intersect (est rows=13) (est cost=3120.355953) over binary (est rows=182) (est cost=6236.583665); left=Some(Exact(13.0)); right=Some(Exact(14.0)); closing=14 (homogeneous proxy); lower bounds must win over their open tails; Sampled { size: 256 }; repeats=11410
  traversal.source "KNOWS" reverse=false: chosen=l2/redb over csr; not built; Exact; repeats=22820
  traversal.source "KNOWS" reverse=true: chosen=l2/redb over csr; background build in progress; Exact; repeats=22820
  expand.intersect two-hop closing domain: chosen=intersect over binary; auto cost admitted; sorted domains; Structural; repeats=11410
  expand.into closing degree: chosen=scan; cached domain length or catalog degree versus one seek; unknown retains scan; Structural; repeats=36430
  …
  expand.into closing degree: chosen=probe; cached domain length or catalog degree versus one seek; unknown retains scan; Structural; repeats=6644
  expand.into repeated binding: chosen=probe over scan; auto cost admitted; fixed bound target; Structural; repeats=6644
  expand.intersect bounded closing domains: chosen=binary (est rows=2080) (est cost=7658.041013999999) over intersect (est rows=13) (est cost=10437.203915000002); left=Some(Exact(13.0)); right=Some(Exact(160.0)); closing=160 (homogeneous proxy); lower bounds must win over their open tails; Sampled { size: 256 }; repeats=1590

• entry.cost would start at b (node[2]). entry.intersect then overrides this to a, which gives the intersection its complete prefix.
• expand.intersect bounded closing domains is the per-candidate sampling decision. With 13 and 14 neighbours the intersection wins. Against a hub with 160 neighbours, the binary plan wins. Both appear, with their own repeats.
• expand.into … probe is the C0 closing probe, admitted where the degree makes a scan dearer than one seek.

With WCOJ=on the cyclic join takes the whole pattern, and the MatchScan becomes unobserved:

Executed operators (query thread):
  [0] Project items=1 distinct=false return_star=false rows=1
    [1] Aggregate group_by=0 aggs=1 rows=1
      [2] MatchScan required=[(a:Person) -(b:Person) -(c:Person) -(a:_)] rows=unobserved (bypassed/fused)
  …
  join.wcoj variable order / (domain, conditional) estimates: [(Some("a"), Some(1000.0), Some(1000.0)), (Some("b"), Some(1000.0), Some(13.0)), (Some("c"), Some(1000.0), Some(0.16899999999999998))]: chosen=leapfrog over binary; completed base adjacency join; Cached; repeats=1

All three runs (off, auto, on) returned 26,760 triangles.

Practical guidance

Make the planner’s job easy

• Use parameters. Parameters share one cached plan and keep plan-build cost off the hot path. Literals are normalized too, but not in queries with two or more MATCH clauses or with comments. Remember that every value of a parameter shares the join order built for the first one.
• Label the anchor and its neighbours. Unlabelled nodes are skipped as entry candidates. Put the most selective label first when you create nodes, because only the primary label has exact counts and value indexes. A label used only as a secondary label is estimated as the whole graph.
• Filter on indexed properties of labelled nodes.
  • Equality, range and STARTS WITH on a property of a node’s primary label can seek the label-scoped value index (see the data model); the IC9 plan of Example B shows (message:Message) → PropertyIndex[creationDate] < est.30000 although the dataset declares no index on creationDate.
  • Declare a compound index for equality plus range or sort combinations.
  • Declare an edge-sorted index for “traversal, then ORDER BY target.prop LIMIT k”. EXPLAIN prints an Advice: line with the exact DDL when such an index would apply.
• Paginate with keys, not deep SKIP. WHERE n.p > $cursor ORDER BY n.p LIMIT k seeks to the cursor. SKIP n still has to walk past n qualifying rows. See Planning note for filtered pages.
• Keep comma patterns connected. A pattern that shares no variable becomes a cartesian product, capped at 10,000 rows by default.
• Avoid literal-only differences in big multi-MATCH queries. Such queries are cached per exact text, so each literal variant builds its own plan.

Diagnosing a slow plan, step by step

1. EXPLAIN first. Check the [entry] marker and the access path of every MatchScan. Warning signs:
   • LabelScan ⚠ (no matching index), or AllVertices ⚠;
   • estimates of est.∞ or of the whole graph (secondary-label fallback);
   • Join without keys (a cartesian product).
2. Read Decisions:. Find the entry.node or entry.cost line for the slow pattern. Default or Stale provenance means the planner had no usable statistics. Check the limit.rule walk and any Advice: line.
3. PROFILE.
   • Compare rows= with est= for every MatchScan and join; the largest q-error is usually the culprit.
   • Check whether expensive operators are unobserved (a fast path engaged), or carry large repeats (seeded per outer row).
   • The traversal.source lines show whether adjacency came from l2, redb or csr.
4. Fix the input before the knob. Add the label, the index or the parameter. Rewrite the query so the selective pattern is connected to the rest.
5. Then try a switch on a test system, comparing PROFILE output and timing with and without it. The RFC 0042 switches are on by default, so the comparison is usually against off.

Known limitations

• The reference join estimate ignores fan-out, and cached join orders are never re-validated (see The plan cache).
• The cost model assumes independent filters and uses mean degrees. Correlated paths and hubs reached through non-literal seeds can be misestimated, as in Example A.
• Statistics are primary-label only, except for the degree catalog.
• The RFC 0042 switches are on by default since 2026-10-17; the measured gain is suite-level, and single queries can be slower (see Measured effects). See Limitations, in particular the sections on cyclic joins and factorized operators.

Measured effects

These are the joint switch-matrix results recorded in RFC 0042 (docs/rfcs/0042-query-engine-modernisation.md, section “Joint switch matrix”, #1442). The ratios are geometric means of warm medians, relative to every switch off; below 1 is faster. They come from a shared host with documented disturbances. Treat them as indicative. They are not a qualification and not a promise for your workload.

allauto is COST_MODEL=join FACTORIZE=on WCOJ=auto CSR=auto. In the same runs:

• FACTORIZE carried most of the Velr gain (A1 0.09).
• WCOJ gave LSQB Q3 0.80–0.81.
• SF1 IC2 stayed 8–18% slower with the cost model, with an identical plan. The targeted IC2 line-2 timing was 9.86 ms off against 12.04 ms allauto: per-query cost-model work, not a plan change. The A2 entry-decision cache below removed it. With it, allauto against off gave Velr 0.544, LSQB 0.878 and SF1 0.995, IC2 0.82/0.81 and IC14 0.80/0.71 (3 rounds; RFC 0042 A6). IC10, IC11 line 3 and IC12 line 3 stayed 3–10% slower (IC11 line 3 at 1.10).
• The maintainer adopted allauto as the default on 2026-10-17 on this result (#1442).

Per case, the A6 comparison that the default was adopted on looks like this. Darker bars changed by at least 3% in the same direction in every round; lighter bars did not.

Figure 4a. Velr per case. A1, A4, A6 and A8 are the cases that RFC 0042 attributes to FACTORIZE (indicative, #1442).

Figure 4b. LSQB per case: Q3, Q4, Q5 and Q8 are faster; no case is consistently slower (indicative, #1442).

Figure 4c. SF1 per case: IC2 and IC14 are faster; IC8 line 3, IC10, IC11 and IC12 line 3 are consistently 3–10% slower (indicative, #1442).

Figure 4d. Leave-one-out on the earlier, pre-fix binary: removing FACTORIZE costs Velr most. The three combined-switch regressions that RFC 0042 records for this run (LSQB Q4 and Q7, SF1 IC8) were fixed afterwards; compare the table above (indicative, #1442).

Planning budget

With a cost model on, the entry model runs every time a pattern is executed, because the entry choice is not part of the cached plan. Three mechanisms keep that cheap without changing any decision (RFC 0042 A6, #1442):

• Catalog reuse. The degree catalog is decoded once per transaction and published catalog, and each label and relationship-type generation is read once.
• Exact pruning. Candidates are priced in ascending order of a lower bound: rows × non-empty sides × one seek. A candidate whose bound already exceeds the best estimate is skipped. EXPLAIN and PROFILE still price and show every candidate.
• A2 entry-decision cache (introduced in RFC 0042 A6). A completed A2 choice is reused by later executions of the same pattern with the same parameter values.
  • The cache holds up to 64 entries per process, within the governor-visible planner.entries cap of min(Q/16, 4 MiB) estimated bytes (#1467). Q is the auxiliary cache quantum after native/CSR adjustments. PATINADB_CACHE_LIMIT=0 disables reuse; there is no separate A2 switch.
  • On a fill, pattern and parameter ownership is estimated and reserved before copying. Oversize keys, values exceeding the 128-level sizing traversal, and function/expression AST literals without a layout model skip this optional cache. The query still uses the ordinary entry planner. Hits retain exact key comparison and snapshot validation without repeating the sizing walk. The byte model includes live construction and evicted-but-pinned readers; it is not an exact allocator or RSS bound.
  • Cache contention causes a miss or skipped fill, so planning uses the ordinary cost model. Snapshot metadata is validated outside the global cache mutex (#1466); every existing dependency check still applies.
  • An entry is reused only while the schema, the statistics, the published degree catalog and every label/relationship-type generation the choice read are unchanged.
  • Transactions that wrote, snapshot views, EXPLAIN/PROFILE and runs that requested a statistics refresh do not use it.
  • A valid hit skips entry costing; validating its metadata still performs reads (RFC 0042 A6). Native READY and generation readers now decode borrowed bytes (#1474), avoiding their owned value copy while keeping all these reads and dependency checks. Overlay reads retain their existing fallback, and buffered own writes can still copy during lookup. No new contention or query-throughput measurement is claimed.

A budget gate that skipped the cost model for cheap-looking queries was measured and not adopted. It judged each inner pattern alone, so a query with an expensive pattern comprehension lost the model’s better entry. See RFC 0042 A6.

Appendix: the example dataset

The outputs above were produced on a graph generated by these statements, executed in order on an empty embedded database. The EXPLAIN and PROFILE runs used PATINADB_CACHE_LIMIT=1GiB, except the one shown without a cache limit. Example D and the factorized, CSR and parallel excerpts also ran under a 2 GiB query memory scope. Each switch combination was a separate process. Every process except the edge-sorted-index run (switches off) ran CALL patinadb.stats.degrees() first.

CREATE INDEX ON :Person(id);
CREATE INDEX ON :Tag(id);
CREATE INDEX ON :Country(id);
CREATE INDEX ON :City(id);
UNWIND range(1, 5) AS i CREATE (:Place:Country {id: i, name: 'country' + toString(i)});
UNWIND range(1, 25) AS i MATCH (co:Country {id: (i % 5) + 1}) CREATE (:Place:City {id: 100 + i, name: 'city' + toString(i)})-[:IS_PART_OF]->(co);
UNWIND range(1, 1000) AS i MATCH (c:City {id: 100 + (i % 25) + 1}) CREATE (:Person {id: i, firstName: ['Chau', 'Ali', 'Jun', 'Eva'][i % 4], lastName: 'L' + toString(i % 17), creationDate: 1300000000000 + i * 1000000})-[:IS_LOCATED_IN]->(c);
MATCH (a:Person) UNWIND [5, 10, 15, 29, 61] AS d MATCH (b:Person {id: ((a.id + d - 1) % 1000) + 1}) CREATE (a)-[:KNOWS {creationDate: 1300000000000 + a.id * d}]->(b);
MATCH (a:Person) WHERE a.id <= 10 UNWIND range(1, 150) AS d MATCH (b:Person {id: ((a.id + d * 3 + 100) % 1000) + 1}) CREATE (a)-[:KNOWS {creationDate: 1300000000000 + d}]->(b);
UNWIND range(1, 40) AS i CREATE (:Tag {id: i, name: 'tag' + toString(i)});
MATCH (p:Person) UNWIND [0, 17] AS k MATCH (t:Tag {id: ((p.id + k) % 40) + 1}) CREATE (p)-[:HAS_INTEREST]->(t);
MATCH (p:Person) UNWIND range(1, 10) AS k CREATE (:Message:Post {id: p.id * 100 + k, creationDate: 1300000000000 + (p.id * 37 + k * 1009) * 100000, content: 'post ' + toString(k)})-[:HAS_CREATOR]->(p);
MATCH (m:Post) WITH m MATCH (t:Tag {id: (m.id % 40) + 1}) CREATE (m)-[:HAS_TAG]->(t);
MATCH (m:Post) UNWIND [3, 11] AS d MATCH (p:Person {id: ((m.id * d) % 1000) + 1}) CREATE (c:Message:Comment {id: m.id * 10 + d, creationDate: m.creationDate + d * 1000, content: 'comment'})-[:HAS_CREATOR]->(p), (c)-[:REPLY_OF]->(m);
MATCH (c:Comment) WITH c MATCH (t:Tag {id: (c.id % 40) + 1}) CREATE (c)-[:HAS_TAG]->(t);
MATCH (m:Message) WHERE m.id % 3 = 0 WITH m MATCH (p:Person {id: ((m.id * 13) % 1000) + 1}) CREATE (p)-[:LIKES]->(m);

KNOWS offsets 5, 10 and 15 form triangles. Persons 1–10 each get 150 extra outgoing KNOWS edges (hubs). Message is the primary label of every post and comment.

Query Planning & Performance

patinaDB compiles each query into a streaming operator tree (the “Op tree”), picks a physical access path for every entry point, and — on skewed data — uses a statistics catalog and a cost model to choose between competing plans. This chapter shows how to read what the planner chose and which fast paths make a query cheap.

Start with How the Query Planner Works. That chapter is the structured reference: the planning pipeline, entry and join choice, statistics, the experimental RFC 0042 switches and how to read the Decisions: section. This chapter keeps the detailed per-feature notes.

Immediate IN operands

When an IN predicate has a single parameter or literal as its right-hand operand, execution borrows that value for the membership check, including inside compound Boolean predicates. The source value remains owned by the statement or plan. This avoids making a new owned RHS copy for every candidate row. Row-dependent RHS expressions continue through the ordinary evaluator. Query memory limits still apply to owned inputs, intermediate work and results; this behavior introduces no configuration switch or higher limit.

WHERE placement

A WHERE comparison of a property with a literal or parameter is checked inside the pattern itself. This applies in either operand order ($d > m.date is the same as m.date < $d), in chained comparisons, and in every MATCH after a WITH. It applies only to the MATCH clauses written after the last UNWIND, CALL or LOAD CSV before the WHERE, so a filter never skips an error raised by such a clause.

WHERE n:Label on a node of a required pattern becomes a pattern label, so the label can choose the scan. A comparison of a single relationship’s property with a literal or parameter (WHERE r.since > $d) is checked while the relationship is expanded; a failure to read that property fails the query rather than dropping the row.

In a MATCH with several comma-separated patterns, a condition that reads only one of the joined patterns runs on that pattern before the next one is expanded. This covers =, <> and IS [NOT] NULL on its variables, properties, literals and parameters, combined with NOT, AND and OR. Conditions that can raise an error stay where they were written.

Reading EXPLAIN and PROFILE

For a direct property projection, RETURN n.key AS key ORDER BY key can use the same ordered property scan as ORDER BY n.key for an eligible single-node shape, subject to index/population safety checks. Traversals, DISTINCT and multi-key ordering keep their previous scan hints and post-sort behavior. Computed aliases such as toLower(n.key) do not imply that the original property’s index order is usable. An ascending STARTS WITH scan can seek to the prefix lower bound while retaining residual filters before applying the output window. Deep SKIP still requires advancing past qualifying rows; use keyset pagination when the application can express a stable cursor. These are generic planner rules, not special cases for ticket IDs.

EXPLAIN renders the chosen plan without running the query; PROFILE runs it and prefixes the elapsed time. For buffered Bolt EXPLAIN/PROFILE requests outside an explicit transaction, the effective client/server query deadline covers the complete execution, including time waiting for a blocking worker. Native execution runs off the async connection thread. Deadline enforcement remains cooperative; it cannot interrupt every individual storage call. No separate profiling timeout is used.

EXPLAIN MATCH (p:Person {name: 'Ada'}) RETURN p

plan: 'Project items=1 distinct=false return_star=false
  MatchScan required=[[entry](p:Person) → PropertyIndex[name] = est.1]
'

Read the tree bottom-up: the MatchScan is the leaf that produces rows, and each line above consumes them. The arrow annotation is the access path and its estimated cardinality — here a PropertyIndex[name] = est.1 point lookup on the value index (one estimated row), not a full label scan.

A range predicate with an ORDER BY that a value index can serve turns into a sorted seek and emits a Physical: footer naming the executed access path:

EXPLAIN MATCH (r:Reader) WHERE r.age > 40 RETURN r.name ORDER BY r.age

plan: 'Project items=1 distinct=false return_star=false
  MatchScan required=[[entry](r:Reader) → PropertyIndex[age] SORTED SCAN ASC est.2]
Physical:
  entry.range_seek
'

The Physical: footer describes the planned physical rules from the shared registry. Runtime admission and preparation can still retain another path; use PROFILE’s executed-access observations to distinguish the actual choice from the plan. Common footer names:

Choosing an equality property

For an unsorted pattern with several eligible equality properties, the property rule compares bounded index probes instead of using the first property in the query. For example, a shared run_id need not drive every lookup when id identifies a much smaller set. The selected predicate and all other conditions are still enforced, including repeated conditions on the same property.

Each candidate probe shares a raw-key budget across its numeric representations and any overlay source and local entries. Probes that reach the limit tie by field name and encoded value; they do not compute a full distribution. The estimates use committed data and can omit pending buffered writes. This can change which plan is preferred, but execution still reads its own writes. These estimates never prove that a result is absent. Existing UUID, IN-list and sorted access rules retain their precedence. EXPLAIN can separately collect its existing cardinality estimates from the statistics catalog.

For label-only entry selection, a zero count in the primary-label index does not prove that a registered label is empty: vertices may carry it as an additional label. In that case the planner uses a coarse source-total estimate, or an unknown estimate when the source total is zero. A label absent from the registry can still estimate zero. This changes plan choice, not matching or result completeness. Positive primary-label estimates are retained and may still underestimate labels that also have secondary holders. There is no new setting for this fallback.

Secondary-label decoding keeps up to 16 successful registry ID-to-label mappings per storage manager in fixed inline slots. A slot collision uses the ordinary registry lookup; missing IDs and errors are not cached. The memo is local to its storage view and remains valid across sync and graph clearing because registry IDs are never recycled. It does not cache a vertex’s current label membership. There is no new configuration setting; query memory limits and error handling are unchanged.

For multi-pattern inner joins, the planner can check an unestimated fixed-length path earlier when at least two distinct nodes in that path are already bound, or when it is a closed path whose starting/closing node is already bound. It uses the existing seeded nested-loop execution instead of building the entire unbound path. The initial driving pattern stays unchanged. Variable-length and open single-anchor paths do not qualify for this rule. This is a heuristic: unknown fan-out can still make a plan expensive, and no latency bound is implied. It adds no setting and does not change matching, edge uniqueness, or query deadlines.

The statistics catalog

The planner keeps a lazy, cached statistics catalog. Per label it holds a vertex count, taken from the primary-label index. Per property it holds column statistics from the value index: the present count, distinct-value count (NDV), min/max and a 16-bucket equi-depth histogram. CALL patinadb.stats computes the same quantities exactly and fresh, including secondary-label holders, so its output can differ from the planner’s cached, primary-label-only estimates:

CALL patinadb.stats('Person')

For a small Person dataset (3 nodes) this yields, per property:

In the procedure output the label summary row has a NULL property. The catalog is computed on demand and cached in RAM only. Each entry is tagged with its label’s write generation and served until it is more than PATINADB_STATS_STALENESS_BUDGET generations (default 64) old. A generation advances once per dirty label per storage synchronisation. Because statistics only change which plan runs, never the result, they are never replicated across a cluster; each node computes its own. The degree catalog and freshness rules are described in How the Query Planner Works.

Cost-based selection

Two things use the catalog:

1. Entry-point selection. For a skewed EQ predicate (a value with many rows) or a range predicate, the estimator tightens its guess using the catalog (present_count / NDV, or histogram interpolation) instead of a pessimistic whole-label count. That can move the entry point of a multi-node pattern to the genuinely cheapest node.

2. Physical-rule ranking. When both a covering compound index and a selective single-property seek apply to the same node, the planner ranks them by an estimated Cost { rows, io } rather than static priority — so a non-selective compound index correctly loses to a selective single-property seek. When the cost model overrides the default pick, EXPLAIN’s footer says so (entry.cost_override: …).

3. Join ordering. A multi-pattern comma-MATCH is reordered so the most selective pattern drives the join and the intermediate result stays small (a bounded Selinger DP for ≤ 8 patterns, greedy above that). Connectivity is preserved, so the reorder never introduces a cartesian product the written order avoided.

On valid data, join order and access-path changes preserve the result multiset. Different access paths can visit different records and change storage-error order, or whether corruption in an unrelated record is observed. Errors from accessed records still propagate. A query without ORDER BY returns rows unordered; one with ORDER BY post-sorts.

Fast paths that make queries cheap

The executor recognizes a number of shapes and serves them without a full scan or a blocking sort. You don’t opt into these — they fire automatically when the shape and the available indexes match. The Physical: footer tells you which fired.

The advisor

When a traversal + ORDER BY target.prop LIMIT k shape could be served by an edge-sorted index but none exists, both EXPLAIN and the advisor procedure tell you exactly which index to create:

EXPLAIN MATCH (c:Company {name:'Acme'})<-[:WORKS_AT]-(p:Person)
RETURN p ORDER BY p.age DESC LIMIT 3

plan: 'Limit skip=0 count=3
  Project items=1 distinct=false return_star=false
    Sort keys=1
      MatchScan required=[(c:Company) -[entry](p:Person) → PropertyIndex[age] SORTED SCAN DESC est.3]
Physical:
  limit.bounded_topk
Advice: CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age  (would serve this ORDER BY … LIMIT via edge_sorted_topk instead of a fan-out top-K)
'

The same suggestion is available programmatically:

CALL patinadb.advisor(
  'MATCH (c:Company {name:"Acme"})<-[:WORKS_AT]-(p:Person) RETURN p ORDER BY p.age DESC LIMIT 3'
) YIELD suggestion

suggestion: 'CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age'

Creating that index flips the plan to limit.edge_sorted_topk and the advice disappears. See Edge-Sorted Indexes.

Bounded-memory guards

Reducible aggregates consume input incrementally; their group state and DISTINCT/collect buffers remain bounded. Blocking operators (non-top-K ORDER BY, UNION-distinct, and hash-join build sides), plus the row expansion of UNWIND inside a correlated subquery, pattern-comprehension result lists, and each partial-prefix sorting group (including the NULL group), are capped at PATINADB_MAX_AGG_ROWS (default 5,000,000). Past the cap they raise a clear, actionable error instead of an OOM — the default is far above any normal query, so below it the result is byte-identical. See Configuration and Caching & Memory Tuning for the RAM budget model, and Cache Observability & Tuning for the read-cache hit rates that make repeated hot queries cheap.

Subquery planning and warnings

Scalar expressions share recursive planning in RETURN, WITH, WHERE and function arguments. This includes COUNT/EXISTS nested in lists, maps, CASE, list comprehensions, REDUCE and ALL/ANY/NONE/SINGLE. Iterator/accumulator variables are local: [x IN [1,2] | x] uses its own x even if an outer x exists. NULL/empty-list quantifier semantics and short-circuiting are preserved.

For eligible single-pattern bodies, COUNT counts matches incrementally and EXISTS stops on a successful match. Complex WITH/RETURN tails can still use bounded materialization; pattern comprehensions collect bounded result lists. Subquery storage errors, query timeouts and memory-limit errors fail the query; they are not zero counts, false existence results or partial lists. Missing properties still yield NULL; failed property reads fail the query, including dynamic/entity-valued access and traversal target-property filters. Unused CASE/coalesce branches do not perform those reads.

EXPLAIN and PROFILE append deduplicated Warning [code] lines:

Eligible repeatable serial projections reuse complete projected rows using the union of their proven input dependencies. Changing an unused outer column does not force another COUNT evaluation. Changing a returned column remains relevant to the key even if the COUNT itself depends on fewer columns. This is whole-row projection reuse, not independent scalar-subquery caching or grouped decorrelation. Unknown dependency forms (including wildcard subplans and opaque expressions) retain the complete typed input key. The existing row and 8 MiB serialized-data budgets apply; key serialization is bounded and does not clone input values.

Within an eligible projection, filter, aggregation, sort or UNWIND source, each COUNT expression may also cache its own integer result using its proven typed input dependencies. This includes graph patterns and trailing subplans, even when unrelated output columns change. Unknown dependency forms keep full input keys. Reuse is scoped to one operator and snapshot; faults are not cached. Volatile or unproven effects disable it. Each retained scalar entry charges its encoded key plus 264 bytes; all scalar entries and equality tables below share one 8 MiB auxiliary accounting budget and entry cap. Oversized keys and exhausted budgets keep ordinary execution; already retained entries remain usable. Different typed keys still execute the inner plan unless the independent equality-scan path below applies; otherwise mostly-distinct inputs can pay overhead without a benefit.

Boolean wrappers also retain native planned contents: XOR and scalar expressions used as predicates do not hide nested COUNT from these contexts. Predicate values must be Boolean or NULL; other types raise an argument-type error. XOR evaluates both operands once, left to right, while enclosing AND/OR and CASE retain their short-circuit behavior. This does not make volatile expressions reusable.

A snapshot-proven projection, filter, aggregation, sort or UNWIND source may also retain equality counts for COUNT { (n) WHERE n.property = key } expressions (either equality order), including nested arithmetic, functions, CASE, lists/maps, indexing/slicing, list predicates/comprehensions, REDUCE and pattern-comprehension expressions. Here key may be a parameter or a snapshot-stable expression over outer bindings, including node/relationship or map properties, arithmetic and containers. The dependency proof must exclude every inner binding; unknown or volatile expressions retain ordinary execution. Direct id()/elementId() comparison operands retain their special validation/coercion path. On an empty inner relation the key is not evaluated, including later table lookups. On the first nonempty scan its evaluation preserves operand error order; later key errors still propagate through the evaluator. The body may be a single-node scan or a graph path such as COUNT { (n)-[:R]->(m) WHERE n.property = key }. Any named node or relationship in that path may supply the compared property; a proven snapshot-stable inner-only expression such as n.v + 1 or n.v + m.v is also eligible. Its dependencies must be known and contained entirely in the inner path. Arithmetic, functions and containers use the existing expression evaluator in original operand order. Expressions mixing inner and outer bindings, volatile expressions and direct id()/elementId() operands retain ordinary execution. A failed evaluation never populates the retained table. Paths may span multiple fixed or variable-length relationships, including zero-length and unbounded ranges. They use the original traversal and preserve path multiplicity; this does not bound the cost of the first traversal. All named nodes, relationships and internal positional path bindings must be independent of outer bindings, with no repeated names, path binding, optional match, dynamic property constraints, explicit seek hints or trailing clauses. Literal and parameter property filters on nodes and relationships are eligible; the original match iterator still applies them. A leading AND prefix such as n.active AND n.v=key is eligible only when its dependencies are proven inner-only and snapshot-stable. It runs before the equality in the original order: false skips both comparison operands, whereas NULL still evaluates them for errors but contributes no count. A prefix error prevents retaining the table. All-false prefixes also skip lookup-key evaluation on reuse. Trailing, volatile or outer-dependent predicates keep ordinary execution; the planner does not reorder them to force admission. Directions and match multiplicity use the same match iterator as ordinary execution. The first required scan computes its normal answer while building auxiliary counts; subsequent supported scalar keys may reuse them even when other output columns change. Empty outer input does no inner scan. Indexed and seeded patterns retain their existing access paths. Complex inner plans retain their normal execution. Each expression uses its current lexical bindings; no COUNT is evaluated before its branch or loop reaches it. All auxiliary equality tables in one operator share the entry and byte budgets below; this does not grant a fresh budget per nested COUNT. Reused lookups retain cancellation checks within a single row.

Filter contexts cover ordinary WHERE, seeded CALL execution, streaming COUNT predicates and both simple EXISTS predicate loops. Hoisted IN membership retains its prepared candidate set and uses the same context for its COUNT needle. An inner loop owns its context for that invocation; it never shares results with another CALL import or snapshot. EXISTS still stops at its first matching candidate, and short-circuited branches are not evaluated early. Volatile predicates disable reuse. Serial aggregation shares its context across group-key computation, aggregate arguments (including collect with entity preservation), and group-key output. All expressions share one auxiliary budget for that operator, rather than one budget per group. Its existing group and collected-value limits still apply. Parallel aggregation gives each worker its own context on the pinned snapshot; group-key finalization uses a separate context on that same snapshot. Budgets apply per worker, with separate later finalization accounting; 8 MiB is not a global query heap limit. No cache crosses workers or query invocations. Full sort and bounded top-K retain one context across key-computation windows, including seeded CALL sorting. Input and all sort expressions must pass the effect proof; unknown callers retain ordinary evaluation. Stable tie order, NULL ordering and the existing heap/input limits are unchanged. Mostly-distinct keys can cost more; use PROFILE and representative measurements. This is bounded reuse, not general decorrelation.

UNWIND prepares function, list and general expression sources once in the plan. Each source keeps a bounded context across seeds within one repeatable APPLY execution; nested invocations and snapshots do not share it. Literal and plain variable sources retain their existing resolution rules. List entity values, NULL expansion, lazy errors and the existing APPLY/UNWIND row caps are preserved. Procedure arguments and unproven effects retain ordinary evaluation.

The table keeps exact integer keys separate from floating-point comparison buckets, preserving existing mixed numeric equality, including large integers. NULL and NaN do not match. Unsupported inner values or exhausted auxiliary budgets discard the table and retain normal COUNT execution; unsupported outer keys use normal execution. The auxiliary budget charges 256 bytes per retained map entry plus string payload, up to 8 MiB and the aggregation row cap (integer keys can require two entries). This is accounting for retained state, not a measured total query heap bound, and is separate from the projection memo. The first scan incurs grouping work even if no later row reuses it. This is a bounded optimization for an admitted shape, not general decorrelation or a cost-based promise. See the measurement ledger.

Independent COUNT scans with additional repeatable predicates can retain their scan rows when an AND branch contains a proven inner/outer equality. The original complete predicate is evaluated for each outer row in source order; trailing errors, NULL reachability and outer-dependent conditions are not reordered. The first evaluation uses ordinary execution. The second may build a bounded scan cache; later evaluations can avoid graph traversal, but still decode rows and evaluate the filter. This is not a hash join or constant-time count lookup. The cache shares the context’s row/8 MiB auxiliary budget, charges serialized payload plus overhead, and is discarded on overflow or evaluation errors. Volatile or unproven effects, correlated scan bindings and unsupported scan shapes retain ordinary execution. Small input counts may not amortize cache construction; measure representative queries.

Individual repeatable EXISTS predicates also retain bounded Boolean results by proven typed dependencies, so changing unrelated outputs need not repeat them. Unknown dependencies use the full input row. The memo is owned by the same projection, filter, aggregation, sort or UNWIND context as COUNT and shares its entry/byte budget. Cache misses preserve the original edge probe, lazy pattern execution or bounded complex subplan; errors are not stored. Hits still check cancellation, and separate CALL invocations, workers, writes and snapshots retain their existing context boundaries. Distinct keys can add overhead without reducing execution. This is bounded reuse, not a streaming rewrite of complex EXISTS tails.

These are static plan diagnostics, including conditional expression branches; they are not measured cost estimates or automatic rewrites. They appear in plan text, not as a new Bolt notification protocol. Check PROFILE on representative data and preserve zero counts, NULLs, grouping, duplicates and ordering. For example, EXISTS { (n:Absent) RETURN count(*) AS c } is true even when there are no matching nodes: the inner aggregate still returns one row. Removing that aggregate changes the answer. Grouping by an inner property instead returns no row on empty input. A projected NULL still counts as a row for EXISTS; DISTINCT and WITH-level SKIP/LIMIT can remove that row. Complex tails retain reached projection errors even after an earlier row could establish existence; an enclosing unchosen CASE branch still skips the entire subquery. Do not drop those projections or stop early without proving error/effect preservation. A missing warning does not prove that every runtime guard admits an optimized path.

For an independent count reused across rows, a unit CALL expresses computing it once explicitly:

CALL { MATCH (m:Metric) RETURN count(*) AS total }
MATCH (n:Customer)
RETURN n, total

A correlated count is not interchangeable with that query. COUNT/EXISTS bodies accept one pattern and optional WHERE followed by WITH/RETURN stages. DISTINCT, filters, aggregation and LIMIT affect the final row count or existence. These stages retain the normal intermediate-result bounds. Eligible repeatable expressions may reuse results for repeated bindings within one operator. Use CALL for other supported inner-query forms, including UNION. General automatic decorrelation remains tracked in #1230.

CALL subqueries can combine RETURN branches with UNION or UNION ALL, including imported outer bindings. UNION removes duplicates; UNION ALL preserves them. When ORDER BY repeats a projected aggregate, its DISTINCT modifier is part of the aggregate identity: count(x) and count(DISTINCT x) may sort differently. The same identity applies inside scalar sort expressions such as abs(count(DISTINCT x)) or count(x)+1. Sorting by the explicit aggregate alias also selects the intended result. Unaliased aggregate output names such as count(n.value) are treated as exact column names when reused in these expressions.

WITH DISTINCT inside CALL applies before later aggregates, and WITH * can carry bindings alongside explicitly calculated aliases. These paths retain bounded intermediate results; using CALL does not imply automatic decorrelation.

Equality filters inside COUNT, EXISTS and pattern comprehensions can use property indexes, including $parameters and bare outer values (n.key = wanted or wanted = n.key). The supported combination is a conjunction of simple equalities. Complex filters and outer property expressions retain their general path; this avoids changing evaluation errors. The filter is still checked after the index lookup. This improves access per outer row and does not imply that the subquery has been decorrelated or cached across rows.

coalesce() evaluates arguments from left to right and stops at the first non-NULL value. A later subquery fallback is therefore not executed when an earlier value is present. False and empty lists count as present values. Errors in an argument that is actually needed still fail the query.

Correlated relationship bindings

COUNT, EXISTS, pattern comprehensions and CALL retain the identity of an outer relationship reused by name in an inner pattern. OPTIONAL MATCH candidate caching includes bound relationships and resolved scalar seek values. Rejected candidates do not consume the materialized-row limit, but still count toward query deadline checks. A relationship identity filter can still require scanning candidate edges; it does not imply general decorrelation or a direct edge seek.

A sorted scan inside correlated CALL also honors SKIP, including parameterized offsets. It applies the offset after binding validation and before bounding the collected rows. This path may visit the skipped candidates, unlike standalone key-only index pagination; an index does not guarantee identical pagination cost inside and outside CALL.

Bounded reuse of repeated CALL inputs

Within one APPLY execution and storage snapshot, deterministic read plans can reuse the result for identical declared CALL imports. Other outer columns do not enter a CALL scope or its memo key. CALL without imports has an empty seed and can reuse one result when outer rows exist. Internal planner APPLY retains the complete outer row as its dependency key. Outer duplicates and ordering remain present in the output; empty inner results remain empty. Keys preserve runtime types and representations, so Integer and Float inputs do not share a memo entry merely because Cypher considers their values equal.

The memo holds at most the configured row limit in entries and result rows, and at most 8 MiB of encoded keys/results. Oversized inputs or a full memo keep ordinary execution. Query deadlines, output limits and evaluation errors still apply. Each APPLY invocation owns its memo; it is not reused across queries, parameter changes or transactions.

Admission uses a snapshot-local effect analysis, separate from cross-transaction cache invalidation rules. It recursively covers read-only COUNT/EXISTS, list predicates/comprehensions, CASE, REDUCE, OPTIONAL MATCH, UNION and nested CALL, including unlabeled nodes and untyped relationships. Executable pattern and seek expressions are checked too. Procedures, writes, external input, volatile functions and unproved expression shapes remain excluded. Shortest-path escape hatches currently retain ordinary execution. An explicit non-wildcard output projection allows proven unused imports to be omitted from the memo key. Unknown dependencies, wildcard output and unsealed output retain all imports. Different required imported values still execute separately; this is not general grouped decorrelation. EXPLAIN/PROFILE identifies that remaining work and suggests filtering the outer input or a semantic-preserving grouped rewrite where appropriate.

When APPLY reuse is excluded, additional plan diagnostics identify the cause:

These messages do not recommend silently replacing per-row randomness, external reads or writes with a once-only operation.

Aggregate expressions in lists, maps, index/slice expressions and CASE conditions are evaluated at their enclosing WITH or RETURN grouping stage. For example, UNWIND [1,2,3] AS x RETURN [count(*)][0] AS c returns one row with c = 3. Aggregates within an inner COUNT/EXISTS query keep the inner scope.

Aggregates in iterator sources belong to that enclosing grouping stage too: UNWIND [1,2,3] AS x RETURN size([y IN collect(x) | y]) AS n returns n = 3. REDUCE may consume an aggregate list and use an aggregated initial value. Local iterator variables are available in the body; other outer references must be valid grouping keys when the expression aggregates.

Comprehensions, REDUCE and ANY/ALL/NONE/SINGLE accept the same scalar source expressions, including direct list parameters, slices, CASE, nested comprehensions and COALESCE. For example, [x IN $items[0..10] | x] does not require an intermediate WITH solely to name its source. NULL and empty-list behavior is unchanged.

Introspection functions evaluate their argument first: keys(box.value) and properties(box.value) inspect the selected value, not box. Entity-valued fields retain their node/relationship identity for id, elementId, labels, type, startNode and endNode.

Runtime argument errors inside iterator bodies fail the query. type() accepts a relationship or NULL; a node or other non-relationship is an argument error. Strict toBoolean/toInteger/toFloat/toString reject list, map and graph-entity/path arguments. Use the corresponding ...OrNull function when NULL on unsupported types is intended. This is distinct from an unparseable conversion string, which can still return NULL.

Inline aggregates in iterator bodies (for example [x IN [1] | count(*)]) are a planning error, including when the input is empty. Compute the aggregate in a preceding WITH and reference its value in the iterator, or use an explicit COUNT/EXISTS subquery with its own grouping scope. Aggregate sources such as [x IN collect(n) | x] remain supported.

Within an aggregating CASE expression, every reference outside aggregate arguments must be a projected grouping key. This applies to WHEN conditions, THEN/ELSE values and simple CASE scrutinees. For example, project x alongside CASE WHEN count(*) > 0 THEN x ELSE 0 END to group by x; without that grouping key the query raises AmbiguousAggregationExpression.

Grouping aliases preserve the input variable’s value inside correlated COUNT/ EXISTS queries, pattern comprehensions, paths and boolean predicates. Local iterator variables retain lexical shadowing. Internal group/aggregate names are chosen to avoid user identifiers; using an alias that matches another input or local variable does not change that variable’s value.

A property-only grouping key such as n.v also retains that property inside correlated scalar subqueries. It does not retain the whole n or unrelated properties: using those in the same aggregating expression raises AmbiguousAggregationExpression. Project the additional grouping key only if its finer grouping is intended; otherwise compute the needed value before the aggregation. Grouping by the whole entity retains its identity and properties.

Dynamic properties inside scalar patterns

COUNT, EXISTS and pattern comprehensions evaluate dynamic pattern properties against the current row, including bindings, property accesses, arithmetic and functions. This also applies when COUNT or EXISTS has a WITH/RETURN tail. Anonymous elements remain anonymous in wildcard output and DISTINCT semantics. For a variable-length relationship, dynamic property conditions apply to every relationship in the path; a zero-length path satisfies that condition vacuously. Unreached CASE branches and empty candidate scans do not evaluate the property expression. Reached evaluation errors propagate. These correctness rules do not imply that a dynamic property pattern is eligible for grouped COUNT reuse.

Dynamic pattern properties

Dynamic node and relationship properties in MATCH are evaluated against the current row, including anonymous patterns such as MATCH (:Item {value: k}). Their internal bindings do not appear in wildcard results or change DISTINCT row identity. In OPTIONAL MATCH these conditions belong to the optional clause: when no candidate satisfies them, the incoming row remains. Variable-length relationship conditions apply to every edge; a zero-edge path has no edge condition to evaluate. As with an explicit WHERE expression, a reached invalid expression reports an error; absent candidates do not evaluate that expression.

RETURN DISTINCT * exposes the bindings still visible after the preceding clauses. ORDER BY can refer to those bindings and their properties, including within scalar subquery expressions. A name removed by WITH or never introduced is still unavailable; an explicit DISTINCT projection retains only its projected columns for ordering.

Filters before later reading clauses

A required MATCH ... WHERE filter is applied before a following MATCH, OPTIONAL MATCH, UNWIND or CALL. Dynamic pattern properties obey the same boundary. A row discarded by the required clause does not evaluate later expressions; an error reached by a surviving row remains an error. Patterns separated by commas within one MATCH are evaluated together before its filter.

A literal equality selecting a single anchor before a separate traversal MATCH can still use a covering edge-sorted index for top-K and receive its index recommendation. Extra anchor predicates and unproven shapes retain ordinary execution; multiple matching anchors retain the existing fallback.

OPTIONAL followed by another reading clause

A following MATCH, UNWIND or CALL consumes the rows produced by the preceding OPTIONAL MATCH, including its NULL bindings. Required matches cannot bind those variables before the optional clause applies its own predicate. A missing optional node followed by a required relationship from that node produces no rows; an independent required pattern can still combine with the NULL row.

Reading order before MATCH

A preceding UNWIND or CALL executes before the subsequent MATCH. For example, UNWIND [1/0] AS k MATCH (n:Missing) RETURN k reports the arithmetic error even when the later label has no matches. Empty prefixes still produce no downstream rows, and duplicate prefix rows retain their multiplicity. Explicit WITH is not required to preserve this boundary.

OPTIONAL clauses with multiple patterns

Comma-separated patterns inside one OPTIONAL MATCH must all match and satisfy the clause’s WHERE. Otherwise the input row survives once, with all newly introduced bindings set to NULL; existing outer bindings retain their values. Two separate OPTIONAL MATCH clauses are evaluated independently in order.

Relationship identity across patterns

Within one MATCH or OPTIONAL MATCH clause, different relationship occurrences must use distinct physical relationships, including across comma-separated patterns, anonymous relationships, named paths and variable-length paths. Separate MATCH clauses may reuse a relationship. A zero-length path contributes no relationships to this check. An OPTIONAL clause without a complete valid match retains its incoming row with new bindings set to NULL.

Internal bindings used for these checks are not exposed by wildcard projections. Repeated occurrences of the same node variable, such as a self-loop’s endpoints, appear once in RETURN * and WITH *, including DISTINCT projections. The existing intermediate row and cartesian-product limits still apply.

The relationship uniqueness check completes at the MATCH clause boundary, before subsequent UNWIND, CALL or OPTIONAL MATCH clauses run. Rows rejected by that check do not evaluate expressions in those later clauses. Errors on valid matching rows remain observable.

Automatic integer covering input (#1399)

A broad plain single-label aggregate reading one integer property automatically uses a covering value-index scan when the shape, row estimate and scratch budget pass admission. Stores/inputs estimated below 4096 rows retain existing access. Configured parallel workers also retain existing aggregate admission; this initial policy does not compare covering against parallel throughput. A bounded external merge restores the original UUID order before the existing evaluator consumes values. This avoids changing SUM intermediate overflow, floating-point expression accumulation, group encounter order or DISTINCT behavior merely because an index sorts by value. Missing properties retain NULL semantics. Original group/buffer limits still apply.

The candidate declines floats (index encoding canonicalizes signed zero), wide values, secondary labels, pending derived indexes, copy-on-read mirrors, selective patterns and unproved expressions. Existing grouped-count index execution keeps precedence. EXPLAIN exposes the selected candidate or its fallback reason, including estimated reorder work and insufficient scratch. Encoded key bytes exclude page overhead; current value-index payloads are empty, and primary I/O costs remain unknown. A logical byte estimate is not a prediction of physical reads. Limited synthetic memory-pressure qualification covers this narrow increment; cost selection remains heuristic and broader predicate/type coverage is follow-up work. PATINADB_READ_COVERING=off disables covering for diagnosis. See the configuration controls.

EXPLAIN describes the proposed access without running the query. PROFILE keeps that pre-execution plan and appends Executed aggregate access (query thread), collected at runtime. For example, eligible execution reports agg.covering_integer_uuid with the actual index-record count and reorder=memory or reorder=external merge. Retained execution reports a covering-decline reason and agg.existing_input (serial aggregate) or (parallel aggregate). Existing grouped/count fast paths report their own agg.* name.

The planned choice can differ from execution: an underestimated scratch budget or unavailable scratch during preparation falls back before emitting rows. PROFILE reports that fallback rather than claiming the proposed covering path ran. Index/storage errors, cancellation and duplicate/orphan identities are not converted into successful results. Diagnostics record at most 64 aggregate events, followed by an omitted-event count; they are not per-worker or per-row timings. For a query without observed aggregation the output says so explicitly. elapsed_ms still measures execution, including abandoned preparation, not EXPLAIN planning.

Bounded preparation for filtered aggregates (#1399)

Eligible residual string predicates can obtain candidate node IDs from the value index before aggregation. This preparation is chosen automatically; existing selective seeks keep precedence. It is separate from the integer covering input and its diagnostic switch.

The filter first reuses a prefix of at most 64 decoded records or 64 KiB of encoded keys, with one final key allowed to cross the byte threshold. Already cached column population and observed key width can predict preparation that would exceed its 64 MiB logical key-work budget. The policy never rebuilds a full statistics catalog just to decide admission. A value-sorted prefix is biased, so its match rate is extrapolated only when the first and last encoded values prove that the column is uniform. Missing column statistics retain runtime bounds; label cardinality is not substituted for property population. Stale estimates can change the selected path but do not supply query results.

For supported COUNT grouping, a small exact set of selected roots can keep ordinary property reads instead of preparing the full grouping column. The current heuristic applies to at most 16 selected roots, an estimated label with at least 1,024 nodes and a ratio greater than 64:1. A bounded width prefix can also reject estimated group preparation exceeding the existing 16 MiB retained-value budget. These thresholds are admission heuristics, not a calibrated I/O price.

EXPLAIN reports a planned bounded prefix probe, available advisory counts and budgets. It does not evaluate the predicate to measure selectivity. PROFILE’s executed-access section reports filter.covered_candidates and, when attempted, filter.group_values, with the selected path or fallback reason. Record/key-byte counters cover consumed preparation entries; they exclude metadata/domain probes, page metadata and physical I/O. Worker-local preparation is not included in the query-thread capture. Copy-on-read overlays and pending indexes retain existing access before creating the covering iterator.

Runtime candidate, retained-value, byte and deadline limits remain active even when estimates are too low. One large encoded value must still be read to inspect it; these bounds do not promise total process RSS. The policy can discard a bounded amount of preparation before fallback and does not guarantee that every filtered query becomes faster.

Inline and external vertex properties (#1399)

The logical primary key still begins with the owner UUID, preserving per-owner prefix reads. Schema 7 keeps canonical values up to 256 serialized bytes inline and places wider canonical values in a property/owner payload table. The owner entry carries a tag. Ordinary point reads and hydration resolve that tag using the same storage snapshot; writes update both tables in one redb transaction. This routing is automatic and has no activation flag. The threshold is a fixed layout heuristic, not a measured optimal boundary for every workload.

Keeping small values apart from wide neighboring values helps workloads that need a small property through primary access. Existing costed index/covering, selective and aggregate choices remain in place. The evaluator still decides which values to read and in which order. A property-ordered scan is not selected: its prototype did not demonstrate enough benefit to justify another access path. Broad reads of the wide values themselves are not guaranteed to improve; warm point/hydration overhead and write/index costs still matter.

EXPLAIN’s Vertex property storage line describes the physical representation, separately from the query operators and storage I/O policy. PROFILE adds Resolved vertex property storage: actual inline/external resolutions and their canonical byte counts on the query thread. These include point and range reads; worker threads, caches and buffered values may bypass them. They are not disk traffic, evaluator-call counts or a total-memory measurement.

The payload table retains canonical bytes, including nested values and exact floating-point representations. Missing external data fails as a storage error; it is not interpreted as a missing property. Value-index keys keep their full ordered value, and repeated wide values are not interned across owners. Schema 6 stores require recreation/reimport; no on-disk compatibility migration or dual decoder is included in this format change.

Wide value-index keys (#1399)

Value indexes keep the encoded property value in the key so equality, prefix, range and ordered scans retain their existing semantics. The authoritative property tree keeps the original value. The secondary payload is empty for both direct setters and batch/rebuild paths.

redb 4.3.0 abbreviates byte-key separators in internal B-tree pages where the neighboring keys permit it. Full keys remain in leaf pages: this is neither string interning nor removal of repeated leaf-key bytes. Long common prefixes can prevent useful shortening, and wide full-index scans still visit wide leaf data. This backend representation is automatic; EXPLAIN and PROFILE keep reporting the same logical access paths. No planner flag enables it.

Smaller indexes do not guarantee fewer physical reads for every access path. Constrained-memory qualification found primary scans that read more from disk with the new file layout, despite similar requested read volume. Account for that tradeoff when measuring a workload dominated by wide primary properties.

The final wide-key investigation retained this representation. A fixed-prefix multimap can save space but may scan a whole suffix group for a selective lookup or late page. A separate prefix dictionary restores suffix seeks, yet adds indirection that regressed distinct-wide reads in the native controls. Grouping complete identical values benefits duplicate-heavy data but does not shorten unique values. These experimental representations are not product options.

The source repository records the method and counterexamples in docs/wide-index-layout-decision-1399.md; earlier access-path work is in docs/read-path-1399-plan.md. This concludes the bounded optimization round, without claiming customer/TB qualification or universal optimality.

Primary-property I/O under memory pressure (#1399)

Logical access and aggregate ordering are unchanged. For an existing Linux store larger in allocated bytes than detected RAM, pure primary-property point reads can use a separate no-read-ahead file handle. This avoids applying the same hint to index/range scans or writes. The policy is selected at open and is reported by EXPLAIN’s Storage: line. Smaller stores and unsupported/unavailable platform facilities retain native I/O; PATINADB_PRIMARY_POINT_IO=off is a diagnostic override requiring reopen.

PROFILE’s Storage I/O footer distinguishes backend requests through the primary handle from native requests on the query thread. It excludes worker threads and redb cache hits and does not measure physical disk bytes. Cold latency can still trade off against speculative reads; this is not a universal scan speedup or a replacement for workload-specific measurements. The counters instrument the Linux file backend only; memory, simulation and other platform backends are uninstrumented, rather than evidence of zero I/O.

Fixed traversal to an already bound node

A fixed, single-type hop whose destination has UUID equality can test the typed edge key directly. Directed and undirected relationship semantics, edge properties, destination labels/properties and relationship uniqueness still apply. Other traversal shapes retain the general iterator. This rule has no configuration option; a speedup for a complete query is not yet established by the current LSQB work (PR #1410).

UUID point entry access also completes at the selected vertex: required labels are checked against that vertex’s primary and secondary labels, followed by all remaining property predicates. It does not append the general secondary label scan. Missing vertices yield no row; errors reading the candidate’s own labels or properties remain errors. Other entry access methods retain their existing candidate enumeration (PR #1410).

For property-free targets, traversal can reuse the primary-label ID already stored in the edge key, including targets without a label constraint and secondary-label checks. Secondary membership still has to be read and validated; this only removes a redundant primary-label lookup. Cached edges without an inline ID and property/multi-label constraints retain the full validation path. The existing optional liveness check remains in effect. No new setting or whole-query speedup guarantee follows from this access refinement.

Counting independent path branches

For eligible count(*) aggregates, agg.disjoint_path_count counts each side of a fixed path around a pivot and multiplies the counts. It avoids constructing every combination of the two sides. Each branch streams; count arithmetic checks for overflow and execution continues to enforce the query deadline.

Explicitly configured parallel query workers retain existing parallel admission. Eligibility requires one required path, no grouping or DISTINCT, and disjoint relationship types across the split. Binding names may not repeat. Optional or variable-length paths, property/seek constraints and path bindings retain the existing execution. A proven filter may move to one factor when every referenced binding is a vertex of that factor. Supported expressions are vertex = / <>, AND, NOT, and bare EXISTS for one fixed typed edge between bound vertices. The EXISTS form permits at most one label constraint per endpoint, but no local bindings, properties, seeks, named edges or additional scope. Cross-factor predicates, functions and other subquery forms retain the existing execution.

For the newly admitted compound or EXISTS predicates, the opposite factor is counted first. If it is empty, the predicate is not evaluated: there are no joined rows on which to evaluate it. Otherwise predicate evaluation retains ordinary bindings and fallible errors propagate immediately, without replay or fallback. Count products and sums remain checked; deadlines and existing limits remain. No isolated speedup is claimed for this extension.

For an eligible pure count factor with a fixed endpoint, typed single-edge EXISTS checks may reuse a neighbor set scoped to that factor. A complete set can prove absence; possible matches still use the ordinary point check. Duplicate paths retain their multiplicities. Set memory is charged to the query budget in addition to the existing pair memo, with a separate bounded scan cap. An incomplete capped scan is discarded and ordinary probes continue. Memory refusal, cancellation and read errors propagate without retry. Only live index views admit this optimization; overlay views retain ordinary probes. No new configuration option is introduced (PR #1410).

EXPLAIN reports the selected pivot as a path-element offset; PROFILE confirms actual use. Selection balances branch lengths without a graph-wide statistics scan. This is a heuristic, not a universal cheapest-plan guarantee. The original LSQB comparison at 928cd9ad completes Q1/Q6 with exact counts; other query regressions remain. This is not general performance qualification (PR #1410).

The development extension agg.grouped_disjoint_path_count applies the same factorization to one vertex grouping key at the pivot and one count(*). It retains the ordinary full-path entry choice, traverses that entire half, and counts its rows by the grouped vertex using the existing bounded aggregate. The opposite half is counted once per source group and its count is multiplied with the source multiplicity using checked arithmetic. Only the pivot binding is retained from each source row; duplicate paths still contribute to the count. Groups without an opposite match are removed before the eager result is exposed.

Existing group/output row bounds remain in force. Intermediate source groups can include vertices with no opposite match, so this strategy can refuse at the group cap even when the final group set would fit. It does not raise the cap or retry through another strategy after a resource error. Filters, scalar or multiple grouping keys, DISTINCT, optional paths and unproved sources retain the ordinary executor. Result order without ORDER BY may change; explicit sorting and limits retain their operators. PROFILE records the chosen entry and source-side grouping. The source-entry correction has focused correctness and work-count controls; no isolated speedup is claimed.

Eligible predicate-free COUNT branches reuse the ordinary traversal state machine while omitting terminal row payloads. Candidate checks, relationship uniqueness, label validation and deadlines still run for each match. Branches that evaluate a row predicate retain their bindings. Repeated bindings, middle-entry products, variable-length paths, optional/path bindings, property/seek constraints and active join guards retain ordinary row emission. This changes materialization, not cardinality or traversal semantics; no configuration toggle is added.

A development COUNT adapter can also compare two uniquely bound vertices for = or <> directly from their UUIDs during successful path emission. It uses the same traversal, target validation and relationship-uniqueness checks, and counts complete matches with the existing overflow checks. Unsupported bindings or expressions retain row evaluation before consuming the iterator. This avoids rows for that proven predicate; it does not skip graph validation or prune paths. Focused and restored Q6 controls plus three genuine mutations passed. The later PROFILE validation also passed the retained path-count tests after adaptive removal. Combined broad source validation passed (PR #1410). Ordinary-binary comparisons of the combined changes are recorded in the LSQB measurement ledger; those bundle measurements do not isolate this adapter’s contribution.

A development scalar COUNT alternative can choose an entry within the shorter independent factor when its primary-label cost estimate is smaller than the original pivot’s. It reuses the grouped admission proof, source_half split and shared traversal, then combines each pivot’s count with its opposite branch using checked arithmetic. Exactly one existing MemoryHashMap<Uuid, i64> retains the counts; its actual table allocation and growth consume statement memory credits. The existing aggregate group cap remains. No representative-row collection or second key set is retained. For a uniquely bound stable vertex, the source factor can emit its UUID directly through the same completed traversal rather than build a binding row only to extract that UUID. Admission requires the existing endpoint/fixed-path proof; unsupported shapes retain row evaluation before consuming input. Match order, duplicates and visited errors are preserved. This later source-emission change passed focused/restored and mutation controls. The later b338 combined-binary comparison includes this change; it does not isolate the gain from UUID emission.

Legacy calls without a statement memory scope retain the streaming path. Group bindings _, _anon and internal property bindings also decline before execution. Once execution starts, visited errors, cancellation and resource refusals propagate without replay. The additional retained groups can cause a memory refusal even where the streaming alternative would have completed.

Primary-label counts do not describe the full secondary-label population and are used only as cost heuristics. Cold planning uses bounded index probes, adding planning work; this does not prove a better traversal order. EXPLAIN shows the enclosing candidate, while successful PROFILE execution records agg.local_anchor_path_count with the chosen pivot and entry. Focused tests, the actual hook-bypass counterexample and restored-source lint checks passed. Combined broad source validation and the b338 ordinary-binary comparison are recorded in the measurement ledger. The comparison is descriptive, uses shared OS cache and includes other changes; it does not establish a universal benefit or performance under RAM pressure (PR #1410).

The experimental bounded suffix-count memo was removed after its ordinary-binary comparison established no benefit. Scalar counts retain direct branch traversal and the general row-elision paths described above. Historical measurements are recorded in docs/lsqb-sf01-coverage-2026-10-11.md; the experiment is not a current execution strategy or configuration option.

Grouping consumes owned evaluated keys without copying their heap-backed payload a second time. Borrowed callers retain copying behavior. Numeric equivalence, NULL grouping, entity identity and first-occurrence output representatives remain unchanged. This removes one copy; it does not make grouping allocation-free or remove its existing memory and group limits. No isolated speedup is claimed for these materialization changes.

Closed paths and endpoint joins

The development planner preserves a join alternative for two fixed paths sharing both distinct endpoint bindings when each relationship has one explicit type and the type sets do not overlap. The normal join optimizer still chooses execution order and hash versus seeded execution. Clause-wide relationship identity checks remain. Open chains and shared, untyped or variable relationship domains keep the existing fold policy. The original LSQB Q2 comparison at 83002192 still selects seeded execution and times out. Preserving a join alternative alone does not establish improved performance (PR #1410).

The single-probe cost comparison no longer selects repeated seeded execution for multi-hop paths with unbound interior vertices: it does not estimate the intermediate traversal work. Such cases retain hash execution unless the separate policy for unknown anchored paths applies. One-edge and fully bound interior paths retain their existing cost comparison. Performance validation of this development refinement is pending (PR #1410).

Streaming APPLY and partial join counts

APPLY streams its results one outer seed at a time when the snapshot-read analysis proves the entire consumer plan repeatable. Recursive iterator construction inherits that decision; unproven procedures inside scalar COUNT/EXISTS keep the eager read boundary. The outer input and each seed’s result are still collected, and nested consumers that require rows still collect them. Cumulative output limits and deadlines remain. Unproven procedures, mutations, external input and volatile expressions keep eager evaluation, including before a downstream LIMIT or early iterator drop. A downstream LIMIT or early drop can avoid evaluating later pure-read seeds; errors or cap breaches confined to those unconsumed seeds are therefore not reported. Errors encountered in the consumed prefix still propagate. This is lazy read evaluation, not preservation of the former eager error timing.

After ordinary storage-aware join ordering, eligible ungrouped count(*) over an inner join can group both inputs by their shared vertex keys, join those counts, and sum their products. This preserves multiplicities while avoiding materializing the corresponding cross-product rows. Scalar keys, DISTINCT, optional matches, user grouping, intervening filters and unproved source plans are excluded. A selected APPLY plan is retained. Existing group and memory caps apply; changed physical retention can change when resource refusal occurs.

Relationship identity guards between comma-separated patterns are omitted when their explicit relationship type domains are disjoint. Overlapping alternatives and wildcard types keep their guards. No graph label names or benchmark query texts select these rules.

Exact join membership during fixed-path traversal

For structurally pure keyed inner joins over a correlated fixed path, the executor can reuse the existing join hash table to reject vertices that cannot contribute a result before expanding their remaining path. The original join still emits every matching row, preserving duplicate witnesses. This requires proven vertex bindings and ordinary variable identity; anonymous internal names, scalar keys, optional paths, variable-length paths and predicate/effectful boundaries retain ordinary execution. No graph label or benchmark name selects this optimization.

The filter shares the join table instead of building another set of keys. Additional fixed control structures are charged to the query memory scope; the existing join-table accounting remains unchanged. At most eight keys and eight filters are admitted; larger shapes use the ordinary executor. There is no configuration toggle. Deadline and memory limits are unchanged. Work pruned before expansion performs no downstream reads, so errors in those unvisited branches are not evaluated. Errors from consumed storage reads still propagate.

The extension join.grouped_identity_membership also checks an identity vertex grouping key between two eligible grouped COUNT(*) inputs. Both inputs must satisfy the fixed-path grouped-count proof, and the inner join must use their single output key. Output aliases are supported; anonymous/internal bindings, inherited membership filters and effectful consumers retain ordinary execution. It borrows the already built right join table while evaluating the left aggregate, after source-side grouping and before counting each group’s opposite branch. Source rows and group buffers are still constructed; only the unmatched opposite-branch work is saved. Counts for retained groups and the original join’s multiplicity are unchanged. A deadline check precedes every membership decision, including empty right inputs. Encountered source, allocation and traversal errors propagate; skipped opposite branches do not evaluate potential storage errors or count overflow.

Fixed-hop storage errors

Fixed-hop traversal preserves upstream adjacency storage and decoding errors through relationship-label and property filters. Such an error fails the query rather than silently removing the affected row. This also applies when grouped COUNT uses the same traversal. Existing property-lookup and missing-target handling are unchanged.

Secondary-label membership scans

Single-label scans of secondary labels avoid temporary decoded ID/label vectors when only membership is needed. Each visited record still validates all label IDs, even after a match, and preserves primary/secondary deduplication. Multi-label scans keep the existing path.

An experiment that propagated required labels between MATCH patterns before plan construction was withdrawn after a measured planning regression (PR #1410); the anchored-cycle policy before it is in effect.

Direct EXISTS for a bound typed edge

An EXISTS over a single fixed typed edge between two uniquely bound vertices uses a direct index seek. It preserves endpoint/label checks, direction and errors from performed reads; unsupported shapes use ordinary execution. The seek may change which unrelated records are visited. Key construction reuses the target primary label just read and validated in the same transaction; it adds no cache and assumes nothing from a label carried in an input row. No setting is added. In the LSQB comparison of this change together with the planner rollback, Q3 recovered while Q8 and Q9 kept their 8 s timeouts; no isolated speedup is claimed. See the LSQB measurement ledger.

Completed-prefix filter pruning

For a pure fixed path with unique bindings, a leading AND condition may depend only on a prefix of the chosen endpoint traversal. After the first complete row has passed the normal path consistency checks, a false leading condition can skip the remaining suffixes of that same prefix. Duplicate prefix visits and physical-edge uniqueness retain the ordinary iterator semantics. UNKNOWN still evaluates the right condition; performed-read and evaluation errors propagate. An empty suffix never causes an earlier predicate evaluation. Pruned unread suffixes need not expose errors that ordinary expansion would have encountered.

Optional or variable paths, repeated bindings, seek hints, inherited join guards, and unproven effectful consumers retain ordinary execution. A middle entry or a condition depending on the final leaf retains the ordinary iteration on the same constructed source. No query name, graph schema, new cache or forced entry order selects this optimization.

Shared mandatory labels and branch access

After selecting ordinary join order and Hash/Apply strategy, the planner can propagate an unambiguous label constraint between occurrences of the same joined vertex within one mandatory MATCH scope. It then refreshes leaf scan strategies before eligible partial COUNT grouping. The rule preserves the selected join tree and does not cross optional matches, dependent predicates, scope rebinding or writes. It has no separate configuration switch.

Label population is only an entry estimate; fewer starting vertices can still produce more traversal work, so this rule does not guarantee a faster plan. Separate MATCH clauses and unsupported scopes retain their existing access planning.

Lazy compound IN membership

A sole immutable parameter or literal IN operand may use a lazily constructed, operator-local membership set when execution first reaches that predicate. AND, OR, XOR and NOT retain the ordinary recursive evaluator: skipped branches build nothing, and lhs/null and sticky-error handling precede membership lookup. Only stable plan predicates participate. Safe scalar candidates reuse the existing numeric-equivalence guard; collection needles or unsupported candidate values retain linear three-valued membership. Row-dependent RHS evaluation is unchanged.

Hash tables and retained keys use the existing query memory admission APIs. Total admitted candidate entries are bounded per operator; admission failure is propagated without replay, and context destruction releases retained owners. This introduces no shared result cache or new configuration flag. No isolated speedup is claimed.

Factor consumers

With PATINADB_FACTORIZE on (the default since 2026-10-17, #1442; off restores the ordinary operators), three more operators can work on a pattern that splits at an interior pivot into two independent branches (RFC 0042 B3–B5, #1442). They read each branch once per pivot instead of building every combination:

• Top-k within a factor. ORDER BY … [SKIP s] LIMIT k, when each sort key belongs to the pivot or to one branch.
  • Each branch is sorted once per pivot, and only combinations that can still enter the page are expanded.
  • The edge-sorted and ordered-frontier paths keep priority.
  • If the page contains tied keys on different rows, the ordinary bounded top-k runs instead, so tie order is unchanged.
• DISTINCT over one factor. RETURN DISTINCT under ORDER BY (with or without LIMIT), and WITH DISTINCT … feeding an ungrouped count, when the projection reads the pivot and at most one branch. The other branch only has to exist, so it stops at its first matching row.
• Counts with a cross-branch comparison. Ungrouped count(*) with one a <> b, NOT a = b or a.p <> b.q (or =) between the branches is counted per pivot by inclusion–exclusion.

A branch with a single element at a pivot is folded into the pivot (flatten). When the factors stay degenerate, for example a single hop from an anchored node, the whole operator runs the ordinary plan instead. That choice is made from the operator’s own input and never changes results (ADR 0019).

PROFILE lists the executed choices under factor.topk, agg.factorized, factor.exists and factor.flatten. EXPLAIN is unchanged.

They have no cost admission: every proven shape engages, with or without the cost model. See limitations.

Factor-local grouping

PATINADB_FACTORIZE (on by default since 2026-10-17, #1442) enables a grouped aggregate for proven independent fixed branches. For two branches around a pivot vertex it reads each branch’s group keys once per pivot and combines the key multisets directly into the group table; the existing ORDER BY and LIMIT still apply afterward. PROFILE records agg.factorized when this path executes. It requires an admitted read memory scope. Unsupported patterns or value domains use the reference aggregate, and factor memory refusal falls back before output.

Branches may share a relationship type when they reach the pivot through it in the same way, for example two comments of one ticket:

MATCH (u1:User)-[:AUTHORED]->(:Comment)<-[:HAS_COMMENT]-(t:Ticket)
      -[:HAS_COMMENT]->(:Comment)<-[:AUTHORED]-(u2:User)
WHERE u1.name < u2.name
RETURN u1.name, u2.name, count(DISTINCT t)

Relationship uniqueness then requires two different comments. The engine checks this per ticket at run time, also when a comment has several authors. Floating-point aggregation and SUM with overflowing positive/negative subtotals are not currently admitted.

With PATINADB_COST_MODEL set to entry or join (the default join), the grouped path engages only when the degree catalog predicts that it is cheaper than the flat aggregate; without a catalog the flat aggregate runs. The catalog comes from the background sample the cost model requests on a miss, or from CALL patinadb.stats.degrees(). With PATINADB_COST_MODEL=off every proven shape engages. The weights are fitted on one benchmark host, so treat them as indicative; see RFC 0042 B1/B2 (#1442).

Procedures (CALL)

patinaDB has an extensible procedure framework. Procedures are invoked with CALL, declare a fixed set of named yield columns, and stream rows back into the query like any other operator. A trailing CALL with no RETURN implicitly returns all yielded columns.

CALL patinadb.engrams() YIELD id, message, timestamp
RETURN id, message ORDER BY timestamp DESC

You can register your own read-only WASM procedures — see WASM Procedures below. Everything under Built-in procedures is a built-in, always-available procedure; user-defined ones are called as CALL wasm.<name>(...) after a CREATE FUNCTION … LANGUAGE wasm DDL statement.

Bulk write clause: CALL { … } IN TRANSACTIONS

A sibling of CALL worth knowing alongside the procedures below: wrapping a mutating subquery in CALL { … } IN TRANSACTIONS [OF n ROW[S]] [ON ERROR {CONTINUE|BREAK|FAIL}] chunks its writes into many small, independently committed transactions instead of one giant one — the batched-ingest answer to a LOAD CSV … CREATE memory cliff.

LOAD CSV WITH HEADERS FROM 'file:///data/accounts.csv' AS row
CALL {
  WITH row
  CREATE (:Account {id: toInteger(row.id), name: row.name})
} IN TRANSACTIONS OF 1000 ROWS

Each chunk is its own engram and, on a cluster, its own Raft entry (⌈rows/n⌉ bounded entries, not one unbounded one); a later chunk sees the writes of earlier ones (read-your-writes). ON ERROR FAIL (default) aborts the whole statement on a failing chunk; CONTINUE skips it and keeps going; BREAK stops after it, keeping earlier chunks. It must be the query’s final clause, and cannot run inside an explicit Bolt BEGIN … COMMIT. See Bulk Loading & Import.

User-defined procedures: WASM functions

Beyond the built-ins below, you can register your own read-only procedures compiled to WebAssembly:

CREATE FUNCTION fib (1) LANGUAGE wasm FROM 'file:///opt/patinadb/fib.wasm' EXPORT 'fib';
CALL wasm.fib(10) YIELD result RETURN result;  -- 55

A WASM function runs deterministically and sandboxed (no filesystem, network, clock, or randomness), which is what makes it safe to replicate across a cluster — CREATE/DROP FUNCTION are DDL that replicate by value (compiled module bytes, not the source path) and ride Raft snapshots. See WASM Procedures for the full registration grammar, the ABI, and the replication/RBAC model.

Built-in procedures

Index readiness

Since 0.16.0, CALL db.awaitIndexes() and CALL db.awaitIndexes(300) are supported. The optional timeout is a non-negative integer number of seconds, defaulting to 300. 0 checks readiness once without waiting. Negative values, null, non-integers and extra arguments are rejected.

The procedure completes without output columns, not with a boolean true; there is no YIELD ready. It preserves incoming rows when followed by another query clause. Ordinary index DDL completes synchronously, and subsequent writes maintain indexes synchronously, so a completed build normally needs no further wait. The procedure checks the local persistent deferred-index-build marker; while it is set, it polls until cleared or the timeout expires. Metadata read errors, query deadlines and shutdown cancellation propagate rather than being reported as success. It does not rebuild indexes itself.

This is a local readiness check for the current synchronous build lifecycle, not a validation scan of every index, a barrier for a concurrently starting DDL, or a readiness guarantee for a raw embedded store being modified without the normal query/write coordination. Future asynchronous index builders must join this readiness contract before being exposed to queries.

It also does not wait for Raft replication or history backfill: use bookmarks for causal reads on another replica. db.awaitIndex(name) is not implemented.

History & diff

See Engrams and Diffs.

Schema introspection

These are read-only and run over every surface (REST, Bolt) via the same CALL path — not a driver/browser-only shim.

Vector / embedding search

Registered under two namespaces, like full-text search:

See Vector Search for index creation, the similarity functions, and the ANN accuracy trade-off.

Full-text search

Registered under two namespaces — the Neo4j-compatible name and a patinaDB alias — so existing Neo4j tooling works unchanged:

queryNodes yields real nodes (not just property maps), so they render as graph nodes in the Neo4j Browser and hydrate correctly through Bolt drivers:

CALL db.index.fulltext.queryNodes('docs', 'graph AND database~')
YIELD node, score
RETURN node.title, score ORDER BY score DESC LIMIT 10

See Full-Text Search for index creation and the query syntax.

Graph algorithms

A small set of read-only graph algorithms run over an in-memory snapshot of the current graph. Each takes an optional node-label and relationship-type projection (pass null to include everything) and yields real nodes plus a per-node result. gds.*.stream aliases are provided for tooling discoverability — note they use patinaDB’s simplified (label?, relType?, config?) signature, not Neo4j GDS’s named-graph-projection API.

CALL patinadb.algo.pageRank('Page', 'LINKS', {iterations: 30})
YIELD node, score
RETURN node.title, score ORDER BY score DESC LIMIT 10

The algorithms materialize the projected graph in RAM (fine up to ~10M edges on one box), run in memory, and yield without mutating anything — so they need no replication and can run on any cluster node. Because PageRank is normalized to sum to 1, its ranking matches Neo4j but its magnitudes differ.

betweenness and closeness follow out-edges (directed shortest paths), matching PageRank/degree; triangleCount and labelPropagation treat edges as undirected. Every algorithm is deterministic — the crux for labelPropagation, which is normally randomized: patinaDB uses a fixed synchronous update with a lowest-label tie-break and a hard iteration cap (iterations/maxIterations are bounded at 1000 so a read call cannot become a compute DoS), so every cluster node computes the same communities. Because betweenness/closeness are O(V·E), they suit small-to-medium projections; the in-memory-only scale ceiling applies. APOC utility procedures and weighted/personalized PageRank variants are planned follow-ons.

What “APOC does not exist here” looks like. patinaDB ships no APOC library. Calling one as a procedure — CALL apoc.help() — fails cleanly with an unknown procedure error. But using an APOC function inside an expression (RETURN apoc.text.join(...), WHERE apoc.coll.contains(...)) currently surfaces a raw parser dump listing the tokens the grammar expected at that point (expected EOI, AND, OR, XOR, …) instead of a dedicated “no such function” error. Both mean the same thing: the function is not available. Check the procedure/function tables in this chapter and rewrite the query against what patinaDB actually ships.

Statistics catalog

CALL patinadb.stats('Person')
YIELD label, property, count, ndv, min, max
RETURN property, count, ndv, min, max ORDER BY ndv DESC

The catalog feeds cost-based entry-point selection: it lets the planner estimate how selective a WHERE/pattern filter is (e.g. a rare property value vs a common one) and drive a query from the cheapest starting point. It also feeds cost-based join ordering: a multi-pattern MATCH (a)…, (b)…, (c)… is reordered so the most selective pattern drives the join and intermediate results stay small, instead of joining patterns in written order. Statistics are local performance hints — they change only which plan runs, never the result (a different join order returns the same rows) — so they are computed lazily per node, cached, and never replicated.

Index advisor

CALL patinadb.advisor(
  'MATCH (c:Company {name:"Acme"})<-[:WORKS_AT]-(p:Person) RETURN p ORDER BY p.age DESC LIMIT 3')
YIELD suggestion, reason, current_plan RETURN suggestion
-- suggestion: 'CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age'

EXPLAIN prints the same advice as an Advice: footer. See Query Planning & Performance and Edge-Sorted Indexes.

Cache observability

CALL patinadb.cache.stats()
YIELD scope, kind, bytes, entries, hit_rate
RETURN scope, kind, bytes, hit_rate ORDER BY bytes DESC

Read-only and node-local. When caching is disabled (the default) it yields zero rows (no error). The same view is available over HTTP as GET /mgmt/cache and as Prometheus gauges on /metrics. For every metric and a tuning playbook see Cache Observability & Tuning.

CSV export

Write graph data out as neo4j-admin-style CSV — the same format GET /mgmt/export uses, so an exported file loads straight back with LOAD CSV/bulk upsert (:ID(uuid) reproduces vertex UUIDs, byte-identical).

-- Whole label + its relationships into a directory:
CALL patinadb.export.csv('Person', '/data/exports/people')
YIELD file, kind, rows RETURN file, kind, rows

-- A selective slice via a query:
CALL patinadb.export.query(
  'MATCH (p:Person) WHERE p.age > 30 RETURN p.name AS name, p.age AS age',
  '/data/exports/over30.csv')

Node files carry an :ID(uuid) column, one typed column per property (name:int / :float / :boolean / :date / :localdatetime, or a bare string column) and a :LABEL column; an absent property is a blank cell. Relationship files carry :START_ID(uuid) / :END_ID(uuid) / :TYPE + typed property columns. Query-export entity cells (nodes/relationships) are rendered as their UUID / type — use patinadb.export.csv for a structural round-trip. Export streams row-by-row (bounded memory) over the current committed graph.

Security note. These procedures write a file on the machine running the query — the same file-write consideration as LOAD CSV reading a file:// URL. On the server both are deny-by-default: an export writes only to directories whitelisted with --allow-export-dir, and any file-I/O query is raised to require the global Admin role (details). The procedures are CSV-only; Parquet/Arrow export goes through GET /mgmt/export?format=parquet|arrow (see REST API).

Availability across interfaces

Procedures work identically over REST and Bolt (including the Neo4j Browser) — the same CALL dispatch serves both. Full-text and vector query procedures read from the local node’s copy of the index, so reads are served without a round trip to the leader.

Degree statistics (RFC 0042 A1)

CALL patinadb.stats.degrees(edgeBudget?) explicitly refreshes the local degree catalog. It yields edge_type, direction (out/in), anchor_label, other_label, edges, anchors (nonzero degree), population (including zero-degree vertices), p50, p99, max, heavy_hitters, provenance, sample_edges and sample_anchors. Both endpoint labels include secondary labels. The cost model (PATINADB_COST_MODEL, default join since 2026-10-17, #1442) uses fresh degree groups and requests a bounded background sample on a miss, so this call is not required; use it for an exact catalog or a warm-up after bulk loads. With PATINADB_COST_MODEL=off the planner does not consume these statistics. See configuration.

The positive integer budget counts directional adjacency entries across both trees. It defaults to PATINADB_FANOUT_RECOMP_MAX_EDGES. Above the budget the scan samples 256 uniformly spaced UUID intervals, assuming uniform UUIDs; structured/nonuniform IDs can bias estimates. provenance is exact or sampled. Counts and histogram buckets are extrapolated for samples; max is the observed maximum, and each reported heavy hitter has its exact observed degree. The SpaceSaving list is a candidate summary, not a guarantee of the global top 16. Percentiles are log2-bucket upper bounds, including zero-degree anchors. Groups absent from the sample are absent from the output, not proven empty.

An exhausted edge/group/memory budget or deadline fails the refresh without publishing a partial catalog. The explicit call can scan vertex membership for population counts. Its cache occupies a separate bounded, governor-visible stats.degree region; cache-disabled calls still return their freshly computed rows. Planner paths never run this explicit refresh. With PATINADB_COST_MODEL set to entry or join (the default), a planner miss can request the separate bounded background sample described in configuration.

patinadb.stats(label?) now includes secondary-label holders in its summary and property rows. Existing primary-label output columns remain unchanged.

Database Ontology View

The ontology view describes what is in one database: its concepts (node labels), the exact label combinations nodes carry, the directed relationship shapes between those combinations, the properties observed on each label and relationship type with their value types, and the constraints and indexes that are actually declared. It is a read-only projection of the current committed state (RFC 0039).

The view is descriptive. An observation never becomes a rule:

• 100 % observed coverage of a property is shown as present on all observed members, never as required. A later write without that property is still valid unless an actual constraint says otherwise.
• Labels that occur together (:Supplier:Preferred) are reported as one label combination. No subclass relation is inferred.
• Observed value types are not a type declaration, and observed endpoints are not a domain/range declaration.
• Only real catalog definitions appear as declared rules. When observed data contradicts a declaration (for example a missing value under an existence constraint), both are shown together with the discrepancy; the data is not changed.

Enabling discovery

Exact discovery reads the whole graph, so it is off by default on a server and in the console’s file mode:

PATINADB_ONTOLOGY_DISCOVERY=on patinadb-raft …

Without it, the procedure fails with patinadb.ontology is disabled on this server. The Rust embedded API Dataset::ontology_view is not affected by this switch.

Querying it

CALL patinadb.ontology() YIELD view RETURN view
CALL patinadb.ontology({mode: 'exact', maxRecords: 10000, maxResultBytes: 4194304})
YIELD view RETURN view

The procedure yields one row whose view column is a structured map (a JSON-LD document), not JSON text. It works over embedded Dataset::query, REST /cypher and Bolt, and every transport returns the same document. The database comes from normal routing (USE <db> or the session database).

Options (all optional): mode must be 'exact'; maxRecords and maxResultBytes may only lower the server limits below. Any other key, or a larger value than the server limit, is an argument error raised before any data is read.

The call must be a standalone read of the current state. These are refused with an explicit error instead of returning something weaker:

• a call per input row (UNWIND … CALL patinadb.ontology()), inside a subquery, or in a writing statement;
• inside an explicit transaction;
• any AS OF call, including AS OF the current HEAD (historical ontology views are a later increment).

Authorization

The view covers the whole database, so it requires unrestricted read access to it (the ordinary Reader role is enough). A principal holding any label-, relationship-type- or relationship-property-scoped grant on that database is refused (Neo.ClientError.Security.Forbidden), even if the relationship strict modes are off. The result is never filtered to an empty or partial ontology, and a refusal carries no label names or counts.

Limits

All three are read once at startup. Exceeding one fails the call with ResourceExhausted; no partial view is returned. Accounting is an estimate, not a hard process-memory limit. Collection time grows with the number of nodes and relationships (every node, property and relationship is read); the size of the result grows with the number of distinct label combinations and property names. Measured examples are in the RFC (tag RFC0039).

Deadlines (--query-timeout) and cancellation stop collection; read faults fail the call.

The JSON-LD document

The document carries an inline @context (no network access needed), an absolute @id naming this view, and an @graph. Its first record is the patina:OntologyView with:

Concepts are rdfs:Class, relationship types and scoped properties are rdf:Property; everything else uses the versioned Patina vocabulary (urn:patinadb:ontology:v1:, never dereferenced). Property observations carry population_count, present_count, missing_count and observed_types (one count per type: Integer, Float, String, Uuid, Boolean, LegacyTimestamp, Date, LocalTime, Time, LocalDateTime, DateTime, Duration, Point, Polygon, List, Map, Path). Node property statistics cover every node carrying the label (primary or secondary); relationship property statistics are type-wide. A stored null counts as missing. Counts above 2^53−1 are written as exact xsd:integer literals. Record IRIs are stable for the same database incarnation and names; they are not node or relationship ids. No property values are included.

Control Panel

The console’s Ontology screen (#/ontology, in the Query section, all connection modes) loads the view only when you press Load ontology — it never scans on navigation. The screen shows the database, snapshot, mode and catalog status; a searchable list of concepts and relationship types; a type-level graph whose vertices are label combinations and whose arrows are relationship shapes with counts; and a detail panel separating observed properties from declared rules. Filtering only changes what is displayed; Download JSON-LD always saves the complete received document. Permission, disabled, unsupported-server and limit failures are shown as such, never as an empty ontology; a failed refresh keeps the previous result, labelled as previous. Each console instance serves one database, so switching databases means opening that database’s console; responses for a previous session or database are discarded.

Not included yet

Historical views and comparison, sampled discovery, SHACL export, MCP access and scoped (filtered) ontologies are not available. See Limitations.

WASM Procedures (User-Defined Functions)

patinaDB lets you register your own read-only procedures compiled to WebAssembly — callable from Cypher as CALL wasm.<name>(args), just like a built-in procedure. This is patinaDB’s user-defined-function story: instead of an ad-hoc scripting language, a UDF is a compiled .wasm module, which means it can be written in any language that compiles to WebAssembly (Rust, C/C++, AssemblyScript, Zig, …) and runs deterministically and sandboxed — the property that makes it safe to replicate to every node in a cluster.

CREATE FUNCTION fib (1) LANGUAGE wasm FROM 'file:///opt/patinadb/fib.wasm' EXPORT 'fib';

CALL wasm.fib(10) YIELD result RETURN result;
-- result: 55

Why WASM

Three engines were evaluated (Rhai, LuaJIT, WASM/wasmtime) for embeddable user-defined logic. WASM won on both axes that matter for a replicated database:

• Speed. Near-native on compute, and the fastest scripted call of the three.
• Determinism by construction. A WASM guest module can be sandboxed so it imports nothing from its host — no clock, no RNG, no filesystem, no network. With host imports denied, the same module given the same input produces the same output on every run and every node, every time. That is the property a Raft-replicated write path requires, and it is why patinaDB doesn’t offer a general scripting language as its UDF surface.

A WASM function is currently read-only: pure compute over its scalar arguments, plus (as of increment 2) bounded point-reads against the graph. It cannot write, and cannot reach the filesystem, network, clock, or any randomness source.

Runtime security updates

The bundled runtime requires Wasmtime 48.0.7 or newer in the 48.x series. This includes the fixes for RUSTSEC-2026-0378 and RUSTSEC-2026-0379. These dependency updates preserve the procedure ABI and sandbox limits; they do not require re-registering stored functions.

Registering a function

CREATE FUNCTION [IF NOT EXISTS] <name> [(<arity>)] LANGUAGE wasm
  FROM '<file-url>' EXPORT '<entry-export-name>'

DROP FUNCTION [IF EXISTS] <name>

SHOW FUNCTIONS

• <name> is a bare identifier (or a backtick-quoted one) and becomes callable as CALL wasm.<name>(...).
• <arity> is optional and documents the expected argument count; it’s not currently enforced beyond what the guest’s ABI decoding does.
• LANGUAGE wasm is required — it’s the only language patinaDB currently compiles, so this is future-proofing the grammar rather than a real choice today.
• FROM '<file-url>' is a file:// URL. The module’s bytes are read once, at CREATE FUNCTION time, compiled, validated (see Sandbox & determinism below), and then stored — the file itself is never consulted again. This matters for two reasons: a reopened database re-activates the function with no dependency on the original file still existing, and on a cluster the compiled module’s bytes (not the path) are what gets replicated — a follower obviously can’t read a file that only exists on the leader’s disk.
• EXPORT '<entry-export-name>' names the guest function to call — see the ABI section below for what that function must look like.
• A module that fails to compile, imports anything not on the allowed list (see below), or doesn’t export the required ABI surface, is rejected at CREATE FUNCTION time with a clear error, and nothing is persisted.

DROP FUNCTION removes the definition and makes wasm.<name> uncallable. SHOW FUNCTIONS lists what’s registered.

A worked example

Here is a minimal, complete fib guest written directly in WebAssembly text format (WAT) — small enough to read end to end, and a realistic shape for whatever toolchain compiles your language of choice to .wasm. Save it as fib.wat, assemble it to fib.wasm with wat2wasm (from WABT), and register it:

(module
  (memory (export "memory") 1)
  (global $bump (mut i32) (i32.const 1024))

  ;; A bump allocator — the host uses this to hand the guest its own args.
  (func $alloc (export "alloc") (param $n i32) (result i32)
    (local $p i32)
    (local.set $p (global.get $bump))
    (global.set $bump (i32.add (global.get $bump) (local.get $n)))
    (local.get $p))

  ;; fib(n): Int -> Int
  (func (export "fib") (param $in i32) (param $len i32) (result i64)
    (local $n i64) (local $a i64) (local $b i64) (local $t i64) (local $i i64)
    (local $out i32)
    (local.set $n (i64.load align=1 (i32.add (local.get $in) (i32.const 5))))
    (local.set $a (i64.const 0))
    (local.set $b (i64.const 1))
    (local.set $i (i64.const 0))
    (block $done
      (loop $loop
        (br_if $done (i64.ge_s (local.get $i) (local.get $n)))
        (local.set $t (i64.add (local.get $a) (local.get $b)))
        (local.set $a (local.get $b))
        (local.set $b (local.get $t))
        (local.set $i (i64.add (local.get $i) (i64.const 1)))
        (br $loop)))
    (local.set $out (call $alloc (i32.const 9)))
    (i32.store8 (local.get $out) (i32.const 1))
    (i64.store align=1 (i32.add (local.get $out) (i32.const 1)) (local.get $a))
    (i64.or (i64.shl (i64.extend_i32_u (local.get $out)) (i64.const 32)) (i64.const 9))))

CREATE FUNCTION fib (1) LANGUAGE wasm FROM 'file:///opt/patinadb/fib.wasm' EXPORT 'fib';

CALL wasm.fib(10) YIELD result RETURN result;
-- result: 55

CALL wasm.fib(20) YIELD result RETURN result;
-- result: 6765

If you’re writing the guest in Rust instead of hand-rolled WAT, the same ABI applies: export memory, alloc, and your entry function with the exact signatures in The ABI below; the wasm32-unknown-unknown target with #[no_std] (or a thin marshalling shim over std) compiles cleanly to a module with no imports.

Reading the graph from a function (increment 2)

A WASM function isn’t limited to its scalar arguments — it can also do bounded point-reads against the graph it’s called from, over a consistent snapshot of the database’s current committed state (HEAD). This lets a function compute over a node’s properties, labels, or neighbours without patinaDB having to expose general graph traversal to the sandbox.

A guest opts in by importing exactly these three host functions from the module "patinadb" (any other import — a WASI clock, env anything — is rejected at CREATE FUNCTION time):

Call one of these by passing a node argument through to your entry function — CALL wasm.readScore(n) evaluates n to the node’s UUID, which the entry function then hands to node_get_property. See crates/patinadb-wasm/examples/guest.wat in the repository for a complete reference guest that imports all three.

Bounded, so a function can’t read the whole graph unboundedly: each host call counts against a per-invocation budget (PATINADB_WASM_MAX_GRAPH_READS, default 100,000 calls) — once exceeded the guest traps with a clean error, never a hang. node_neighbors is additionally capped at a maximum number of returned neighbour ids per call. A malformed request (an out-of-bounds pointer, an unknown direction, or calling these outside of a query context) is also a clean trap, never a panic or a crash.

Still read-only: none of these host functions can create, update, or delete anything.

Determinism & the sandbox

Pure computation is deterministic for the same module and arguments. Graph-reading procedures additionally depend on the pinned committed graph state; identical arguments on replicas at different applied positions need not return the same value. Runtime safety limits can also reject a call rather than return a value. patinaDB enforces the following sandbox boundaries:

• No host imports beyond the documented allow-list. A guest module is instantiated with an empty import list by default; the only imports ever accepted are the three graph-read functions above, and only when a graph context is available. Anything else — a WASI clock, a random-number source, a filesystem call, an environment-variable read — is rejected at registration, before the module is ever run.
• Fuel is the deterministic resource bound. Every invocation gets a fixed instruction budget (100,000,000 units by default). The same module and input consume the same fuel and trap at the same point on every node — this is what makes an accidental infinite loop safe to replicate rather than a cluster-wedging hang.
• Wall-clock (epoch) and memory are safety nets, not the primary bound. A 5-second wall-clock deadline and a 16 MiB linear-memory cap protect a single node against a runaway guest, but neither is used to gate a replicated write, because wall-clock timing is inherently nondeterministic across machines.
• The table-element count is capped too (65,536 entries). A guest’s call_indirect dispatch table is separate memory, so the linear-memory cap alone never governed it; without a ceiling a module could declare a billions-of-entries table and allocate far past the 16 MiB cap. Like memory, this is a fixed limit applied identically on every node, so a module either instantiates everywhere or nowhere.
• What the guest hands back is validated before the host acts on it. The returned result blob’s pointer and length are checked against the guest’s own linear memory before the host allocates a buffer for it, and the blob’s nesting is depth-bounded (PATINADB_WASM_MAX_DECODE_DEPTH, default 256) because the host-side decoder runs after the guest call has returned — where fuel and the wall-clock deadline no longer apply. Either violation is a clean procedure error.
• Strict IEEE-754 floating point, with no relaxed-SIMD and NaN canonicalization enabled, so floating-point results are bit-identical across platforms too.

A guest that traps (fuel exhausted, memory cap hit, deadline exceeded, out-of-bounds access, or an explicit guest panic) surfaces as a clean query error — never a process crash or a hang.

Because of this sandboxing, patinadb-wasm is an isolated, opt-in crate: core patinadb has no dependency on it and pulls in no WebAssembly runtime by default. A server or embedding application opts in explicitly (see below); a plain build of the core library is unaffected either way.

Running on the server (replication & RBAC)

CREATE/DROP FUNCTION are DDL, and on a clustered server they behave like any other schema change:

• Requires the global admin role. CREATE FUNCTION reads a file off the leader’s local disk, so it’s authorized like LOAD CSV and the CSV export procedures — by effect, not by role level. A per-database Writer is refused; only a global Admin can register or drop a function. The file path is confined by the same --allow-csv-dir sandbox LOAD CSV uses (see Cypher-driven file I/O).
• Replicated by value, not by path. The leader reads, validates, and compiles the module exactly once, then proposes the resulting definition — module bytes included — as a single Raft log entry. Every node applies it identically: persists the definition and, if it has WASM support installed (see below), activates it. A node without WASM support still stores the definition (so it stays consistent and forwards writes correctly) but can’t execute the function locally.
• Snapshot-carried. A node that bootstraps purely from a streamed Raft snapshot — with the original CREATE FUNCTION log entry long since purged — still ends up with every registered function, because the snapshot itself carries the compiled module bytes.
• Module-size cap. A registered module is capped at PATINADB_MAX_WASM_MODULE_BYTES (default 8 MiB) so a large module can’t blow past the Raft peer-RPC body-size limit. CREATE FUNCTION fails loudly, before anything is proposed, if the module is over the cap.

A server enables this feature with one line at startup (patinadb_wasm::install(), already wired into the shipped patinadb-raft binary) plus a build-time dependency on the patinadb-wasm crate — core patinadb remains WASM-free either way.

The ABI (for guest authors)

A guest module must export:

• memory — its linear memory.
• alloc(len: i32) -> i32 — allocate len bytes and return the pointer. Any allocation strategy works (the reference guest above uses a trivial bump allocator); the host calls this to place argument bytes into the guest’s memory before invoking the entry function, and the guest calls it itself to place its result.
• The entry function named in EXPORT '<name>', with signature (in_ptr: i32, in_len: i32) -> i64. The return value packs a pointer and a length into one 64-bit integer: (out_ptr as u64) << 32 | out_len as u64.

Wire format — a small, frozen, tag-prefixed binary encoding, deliberately not patinaDB’s internal storage format, so it stays a stable, language-agnostic contract for any guest toolchain:

• The argument blob at in_ptr is u32 count (little-endian) followed by count tag-prefixed values.

• The result blob the guest allocates and returns is exactly one tag-prefixed value.

• Each tagged value is one byte for the tag, then the payload:

  Tag	Type	Payload
  0	Null	(none)
  1	Integer	i64, little-endian
  2	Float	f64 bits, little-endian
  3	Bool	one byte, 0/1
  4	String	u32 length + UTF-8 bytes
  5	List	u32 count + that many tagged values
  6	Uuid	16 raw bytes (a node/relationship handle)
  7	Map	u32 count + that many (u32 key-len + key bytes + tagged value) entries
  8	Node	a Uuid + a labels-List + a properties-Map
  9	Relationship	a start Uuid + an end Uuid + a type-String + a properties-Map

A Cypher value outside this set — a temporal value, a spatial Point/Polygon, or a multi-hop Path — is rejected loudly as an unsupported argument/result type rather than silently coerced.

Limitations

Calling from an application backend

Requires 0.21.0: correlated procedure calls after WITH preserve the preceding row bindings. The published 0.20.0 can lose those bindings in this shape. The application example below requires the seeded-procedure fix; it is not evidence that the old release supports it.

Register the module through a separate administrator connection. The backend’s technical user does not need administrator rights to call a registered read-only procedure. Do not accept a module path, arbitrary procedure name or Cypher text from an application request.

For a paginated endpoint, select and authorize the nodes before invoking the guest, then explicitly order the returned rows. For example, with $projects derived from the authenticated user’s backend ACL rather than client input:

MATCH (ticket:Ticket)
WHERE ticket.project IN $projects
WITH ticket ORDER BY ticket.created_at DESC, ticket.issue_key ASC
SKIP $skip LIMIT $limit
CALL wasm.readSummary(ticket) YIELD result
RETURN ticket.issue_key AS key, result AS summary
ORDER BY ticket.created_at DESC, ticket.issue_key ASC

The operator supplies readSummary; it is not a built-in function. Its guest can call node_get_property with the property name summary and forward the packed result. Graph-reading determinism requires the same graph snapshot as well as the same module and arguments. A later call after a committed update can return a different value. Use the normal causal bookmark contract when a reader must observe a preceding write; WASM does not provide a separate replication barrier.

The guest’s graph access does not implement your application’s custom ACLs. Only install trusted modules whose reads respect the backend’s authorization boundary. A guest that traverses from an authorized node to arbitrary other nodes needs a separate authorization design; filtering its initial argument alone does not establish that all traversed data is authorized.

During rollout, keep guest traffic in a separately measured workload lane. Stopping that lane after errors may preserve other customer traffic, but is not a successful WASM qualification. Retain errors and distinguish unexpected query failures from intentionally tested sandbox traps.

Engine limits

• Read-only. A WASM procedure cannot create, update, or delete graph data.
• Not a scalar UDF (yet). A WASM function is a procedure, called with CALL wasm.<name>(...) YIELD result — it does not (yet) compose into an arbitrary expression position like RETURN f(n.x) or a WHERE clause.
• No filesystem, network, clock, or randomness inside the guest, ever — this is the determinism guarantee, not a configurable option.
• The scalar/entity ABI subset above is everything. Temporal values, spatial Point/Polygon, and multi-hop paths are not (yet) marshalled across the boundary.
• Graph reads are single-vertex point-reads and bounded neighbour lists — there’s no general traversal or query-execution host call from inside a guest.
• CREATE EDGE SORTED INDEX-style module toolchains aren’t required or assumed — you bring your own WASM compiler; patinaDB only defines the ABI a compiled module must satisfy.

The full design write-up — including the alternatives considered and the increment-by-increment implementation history — lives in the repository’s docs/rfcs/0017-wasm-embedded-procedures.md.

Full-Text Search

patinaDB supports user-defined full-text indexes with BM25 ranking, using Neo4j-compatible syntax. You create an index over chosen string properties of a label (nodes) or relationship type (edges), then query it through the full-text procedures, getting back ranked entities and relevance scores.

Creating an index

-- Over node properties
CREATE FULLTEXT INDEX docs FOR (n:Doc) ON EACH [n.title, n.body]

-- Over relationship properties
CREATE FULLTEXT INDEX mentions FOR ()-[r:MENTIONS]-() ON EACH [r.context]

-- With an analyzer
CREATE FULLTEXT INDEX articles FOR (n:Article) ON EACH [n.body]
OPTIONS { indexConfig: { `fulltext.analyzer`: "english" } }

• Only string properties are indexed (matching Neo4j). Non-string values on the listed properties are ignored.
• The index is maintained synchronously: it is updated as part of every committed write, so it is never stale on read (no eventual consistency).

Analyzers

The analyzer controls tokenization and stemming:

Managing indexes

SHOW FULLTEXT INDEXES
DROP INDEX docs

DROP FULLTEXT INDEX <name> and DROP INDEX <name> drop a full-text index. An unknown name is an error; add IF EXISTS for an explicit no-op when absent. Malformed trailing text is rejected without deleting the index. Registered ordinary indexes take precedence for the shorthand DROP INDEX <name>; DROP INDEX ON :Label(prop) remains the ordinary-index form.

Querying

CALL db.index.fulltext.queryNodes('docs', 'graph database')
YIELD node, score
RETURN node, score ORDER BY score DESC

CALL db.index.fulltext.queryRelationships('mentions', '"exact phrase"')
YIELD relationship, score
RETURN relationship, score

The alias patinadb.fulltext.queryNodes / …queryRelationships is equivalent. Results are ranked by BM25 (k1 = 1.2, b = 0.75).

Query syntax (Lucene subset)

Prefix and fuzzy queries expand against the index dictionary and are capped at 256 expansions per term to bound cost.

In the server

CREATE FULLTEXT INDEX / DROP INDEX are replicated as Raft control commands (like CREATE DATABASE): the definition propagates to every node and each node builds the index from its own copy of the graph. Index definitions are carried in Raft snapshots and rebuilt on snapshot install, so a node that joins or restarts ends up with the same indexes. Query procedures read the local node’s index.

This works end-to-end over Bolt, including the Neo4j Browser: create an index and run queryNodes, and matching nodes come back as graph nodes with scores.

Limitations

Measured Jira-summary workload (0.17.0)

On 334,770 real Apache Jira summaries, standard analyzer, one indexed summary property, local Ryzen 7 3800X/NVMe, the 2026-09-10 measurement under #1261 found:

• Seventeen tested term/boolean/phrase/prefix/fuzzy/project-filtered forms had warm Bolt Top-10/20 p90 between 2.173 and 63.868 ms (20 samples per cell).
• A short 2 reads/s plus 0.1 summary updates/s run had read p90 66.816 ms and p99 122.826 ms; updates, batch changes, graceful reopen and deletion passed search-visibility checks. Exact hit sets and single-term BM25 scores were independently checked.
• Server high-water RSS reached 378.957 MiB through the batch phase; the standalone build peaked at 464.113 MiB. These exclude the Python oracle.
• Initial construction took 7.950 seconds, versus 1727.563 seconds in the retained 0.16.0 baseline: 217.31 times faster on this fixture (#1261). Allocated database-directory size after building fell from 1375772672 to 477433856 bytes; this is the whole directory, not posting-only storage.

These are summary-only, single-node measurements, not 10M-ticket, multi-region, long-soak, ticket-body/comment or other-analyzer guarantees. Full methodology and raw artifact locations are recorded in the repository’s docs/fulltext-benchmark-1261.md (#1261).

Construction and resource use

Initial construction and reconstruction stage postings into sorted runs and write each term’s final list once. The existing format and synchronous DDL boundary remain unchanged; updates after construction still use per-document maintenance. Existing indexes remain compatible without rebuilding.

The builder uses the sort-run byte setting and private temporary files under TMPDIR. Budget additional memory for merge readers, the largest posting list, serialization and database caches; the run buffer cap is not a total-RAM ceiling. Scratch files are removed on ordinary completion/error/cancellation, but a hard process kill can leave temporary files.

Feature boundaries

• No phrase slop / proximity ("a b"~3) — phrases must be exactly consecutive.
• No highlighting / snippet extraction.
• No numeric or range queries inside the full-text string (full-text indexes cover string properties only).
• Postings are updated read-modify-write per document with no segment merging, so very high write throughput on a large indexed corpus is not the design target. See Limitations.

Vector Search

patinaDB supports vector / embedding search with Neo4j-compatible syntax, backed by an IVF-Flat (inverted-file, k-means clustering) approximate nearest-neighbour index. You store embeddings as list-of-float properties, create a vector index over them, then query it through the vector procedures, getting back ranked nodes and normalized similarity scores.

Storing vectors

A vector is an ordinary list-of-numbers property. Set it with plain Cypher:

CREATE (n:Product {id: 1, embedding: [0.12, -0.03, 0.88, 0.41]})
MATCH (n:Product {id: 1}) SET n.embedding = [0.10, -0.01, 0.90, 0.40]

When a vector index covers that (label, property), the index is maintained synchronously on every write — inserts, updates, and deletes are reflected immediately (no eventual consistency), on every replica.

Creating an index

Since 0.16.0, PATINADB_VECTOR_BUILD_THREADS controls a shared, bounded training pool (1–8 threads, 1 for serial). Initialization caches nearest scores and parallel assignment preserves ordered accumulation. See 0.16.0 Release Notes for defaults and validation scope. This is not a setting for concurrent redb writers or a total-RAM limit.

CREATE VECTOR INDEX product_embeddings
FOR (n:Product) ON (n.embedding)
OPTIONS { indexConfig: {
  `vector.dimensions`: 4,
  `vector.similarity_function`: 'cosine'
} }

• `vector.dimensions` (required) — the fixed vector length. Vectors of a different length, or non-numeric lists, are simply not indexed.
• `vector.similarity_function` — 'cosine' (default) or 'euclidean'.
• IF NOT EXISTS is supported.
• Unknown option keys (e.g. Neo4j’s HNSW-specific params) are accepted and ignored, so existing Neo4j DDL runs unchanged.

Development behavior (not yet released): Training preparation cooperates with the configured server query timeout, including waiting for database write admission. A timeout during preparation returns an error before the new definition is proposed to Raft; it does not roll back a definition that has already been committed. Stored-data read errors fail the build instead of silently omitting unreadable vertices.

A definition becomes visible only after its posting lists have been built. Rebuilding an existing name first withdraws that definition. If backfill fails, the index remains unavailable until the underlying error is resolved and the build is retried; the previous index is not restored automatically. Use IF NOT EXISTS when an existing index should be left untouched.

IVF tuning (patinaDB extensions)

The index partitions vectors into nlist clusters (via k-means); a query scans the nprobe clusters nearest the query vector. Defaults: nlist = clamp(round(sqrt(count)), 1, 4096), nprobe = clamp(nlist / 16, 1, 64). Override them:

OPTIONS { indexConfig: {
  `vector.dimensions`: 384,
  `vector.similarity_function`: 'cosine',
  `patinadb.ivf.lists`: 256,
  `patinadb.ivf.nprobe`: 16
} }

Higher nprobe → higher recall, more work per query. nprobe = nlist scans every cluster (exact search).

Prefer building the index after loading data. The k-means centroids are trained from the vectors present at CREATE VECTOR INDEX time — training over an empty label produces an index with no centroids (there is nothing to cluster over an empty set). If that happens, patinaDB automatically retrains the index the next time a write touches that label, so it self-heals without needing a manual DROP/re-CREATE; the query that lands on the still-empty index right after CREATE (before the first later write) simply doesn’t match anything yet. Building after the data is loaded still avoids that retraining round-trip and remains the recommended order.

Managing indexes

SHOW VECTOR INDEXES
DROP VECTOR INDEX product_embeddings
DROP VECTOR INDEX product_embeddings IF EXISTS

SHOW VECTOR INDEXES yields name, label, property, dimensions, similarityFunction, lists, and nprobe.

Querying

CALL db.index.vector.queryNodes('product_embeddings', 5, [0.1, -0.02, 0.9, 0.4])
YIELD node, score
RETURN node, score ORDER BY score DESC

Arguments: the index name, the number of nearest neighbours k, and the query vector. Results are real graph nodes plus a score, ordered by descending similarity. The alias patinadb.vector.queryNodes is equivalent. A query vector whose length differs from the index’s dimensions is an error.

Similarity scalar functions

Two namespaced functions score a pair of vectors directly, using the same normalized formulas as the index score:

RETURN vector.similarity.cosine([1.0, 0.0, 0.0], [1.0, 0.0, 0.0])      -- 1.0
RETURN vector.similarity.euclidean([0.0, 0.0], [3.0, 4.0])            -- 1/26

• vector.similarity.cosine(a, b) → Neo4j-normalized cosine (1 + rawCosine) / 2 ∈ [0, 1].
• vector.similarity.euclidean(a, b) → 1 / (1 + euclideanDistanceSquared) ∈ (0, 1].

Both raise an error on a dimension mismatch or a non-numeric element, and propagate null when either argument is null.

In the server (High Availability)

CREATE / DROP VECTOR INDEX are replicated as Raft control commands. This is where the IVF choice matters:

patinaDB replicates a deterministic apply loop — every node applies the same operations and must converge to a byte-identical index, or the same query would return different results on different replicas. HNSW is randomized and insertion-order-dependent, so it would diverge. IVF-Flat’s k-means centroids are trained once on the leader and replicated as part of the index definition. Every node then assigns each vector to the nearest centroid by a pure, deterministic function → identical posting lists.

Index definitions (including their centroids) are carried in Raft snapshots, so a node that joins or restarts rebuilds identical posting lists from its restored graph. Ongoing writes are indexed synchronously on every node as they apply.

Works end-to-end over Bolt (including the Neo4j Browser): create an index and run db.index.vector.queryNodes, and matching nodes come back as graph nodes with scores.

How it works / limitations

• IVF-Flat, not HNSW. Vectors are partitioned into nlist k-means clusters; a query scans only the nprobe clusters nearest the query, then computes the exact similarity to each candidate and keeps a bounded top-k. This is approximate: recall is tunable via nprobe (higher = better recall, more work; nprobe = nlist is exact).
• Heavy one-time build. Assigning every existing vector to a centroid is O(N · nlist · dim) over the whole label — inherent to any ANN build. Incremental maintenance per write is cheap (one nearest-centroid assignment + a few KV ops).
• Brute-force candidate scan. Within the probed clusters the candidate scan is a linear posting-list scan. It is deliberately isolated behind one internal function (VectorIndex::candidates), so a graph-based backend (e.g. HNSW per cluster) could replace it later without changing the query surface — but that backend would need to preserve cross-replica determinism.
• Nodes only (v1). Vector indexes cover node properties; relationship vector indexes are not yet supported.
• Build after loading (see the note under Creating an index).

Spatial / Geo

patinaDB has a first-class Point type with Neo4j’s four coordinate reference systems (CRSs), the point() / distance() / point.distance.ellipsoidal() / point.withinBBox() functions, and a CREATE POINT INDEX statement. Points are stored, indexed on disk with a space-filling-curve key, and queried correctly, and a planner fast path turns a radius/bbox query (and a kNN ORDER BY distance(...) LIMIT k) into a curve-range seek instead of a full scan.

The Point type

A point carries a CRS (identified by a Neo4j SRID) and 2 or 3 coordinates:

For geographic (wgs-84) points, x is the longitude and y is the latitude (Neo4j’s convention), so p.latitude reads the second coordinate.

Two points are equal iff they have the same SRID and coordinates — a cartesian and a wgs-84 point are never equal even with identical numbers. < / > on points are undefined (they return null, matching Neo4j); ORDER BY uses a deterministic total order (the space-filling-curve order).

point({...}) — constructing points

CRS is inferred from the map keys, or set explicitly with crs / srid:

// cartesian (x/y) — 2D and 3D
RETURN point({x: 3.0, y: 4.0})
RETURN point({x: 1.0, y: 2.0, z: 3.0})

// geographic (latitude/longitude) — 2D and 3D
RETURN point({latitude: 52.52, longitude: 13.405})
RETURN point({latitude: 52.52, longitude: 13.405, height: 100.0})

// explicit override
RETURN point({x: 13.4, y: 52.5, crs: 'wgs-84'})
RETURN point({x: 1.0, y: 2.0, srid: 4326})

Out-of-range latitude (|lat| > 90) or longitude (|lon| > 180) raises an ArgumentError. point(null) returns null.

Points can be stored on nodes and relationships like any other property:

CREATE (:City {name: 'Berlin', loc: point({latitude: 52.52, longitude: 13.405})})

Accessors

Field access reads a coordinate or CRS component:

WITH point({x: 3.0, y: 4.0, z: 5.0}) AS p
RETURN p.x, p.y, p.z, p.crs, p.srid

MATCH (c:City)
RETURN c.loc.latitude, c.loc.longitude

.x/.longitude → axis 0, .y/.latitude → axis 1, .z/.height → axis 2, .crs → the CRS name string, .srid → the integer SRID. Accessing a missing component (e.g. .z on a 2-D point) returns null.

distance() / point.distance()

Both spellings work. Returns metres for geographic points (spherical Haversine) and Euclidean distance for cartesian points:

// Euclidean → 5.0
RETURN distance(point({x: 0, y: 0}), point({x: 3, y: 4}))

// Haversine, Berlin → Paris ≈ 878 km (metres)
RETURN point.distance(
  point({latitude: 52.52,  longitude: 13.405}),
  point({latitude: 48.8566, longitude: 2.3522})
)

// radius query (served by a curve-range seek when a POINT INDEX exists)
MATCH (c:City)
WHERE distance(c.loc, point({latitude: 52.5, longitude: 13.4})) < 5000
RETURN c.name

Mixed CRS (or mismatched dimensionality) returns null, matching Neo4j. wgs-84 distance is spherical (mean Earth radius 6 371 009 m), ~0.3 % off the true geoid — fine for radius search, not survey work.

point.distance.ellipsoidal() — accurate WGS-84 geodesic

When you need survey-grade accuracy, use the ellipsoidal variant, which computes the true WGS-84 geodesic distance via the Vincenty-inverse formula (accounting for the Earth’s oblateness). distance() stays spherical Haversine (Neo4j parity); point.distance.ellipsoidal() is the distinct, more-accurate spelling:

// WGS-84 ellipsoidal (Vincenty), Berlin → Paris ≈ 878 km — a distinct value
// from Haversine, but within 0.5 %.
RETURN point.distance.ellipsoidal(
  point({latitude: 52.52,  longitude: 13.405}),
  point({latitude: 48.8566, longitude: 2.3522})
)

Same argument conventions as distance(): null/non-point operand → null, mixed CRS/dimensionality → null. It is WGS-84 only — a cartesian argument falls back to plain Euclidean (there is no ellipsoid without a geoid). Near-antipodal pairs (where Vincenty does not converge) fall back to the always-finite spherical distance rather than emitting NaN. It is an exact scalar only — it does not drive the radius/bbox seek (which uses the spherical bounding math); a radius filter should still use distance().

point.withinBBox()

Bool — whether a point lies inside an axis-aligned bounding box:

MATCH (c:City)
WHERE point.withinBBox(c.loc,
        point({x: -1, y: -1}),
        point({x: 10, y: 10}))
RETURN c.name

A geographic box whose lowerLeft.longitude > upperRight.longitude wraps the antimeridian and is handled correctly — it covers [lowerLeft.longitude, 180] ∪ [-180, upperRight.longitude] rather than being rejected or silently returning false.

CREATE POINT INDEX

A point index is a planner-enablement marker. The on-disk curve keys are written for every point property unconditionally (see below), so registering an index is a lightweight no-op backfill; the def tells the planner it may use a curve-range seek for radius/bbox queries over that property (increment 3).

CREATE POINT INDEX city_loc [IF NOT EXISTS] FOR (n:City) ON (n.loc)
DROP POINT INDEX city_loc [IF EXISTS]
SHOW POINT INDEXES        -- also folds into SHOW INDEXES

Like every other index, it replicates across a Raft cluster (re-run deterministically on each node) and is carried in snapshots.

How points are stored (the curve key)

Every point property lands in the same label-scoped value index as every other scalar, under an order-preserving Morton / Z-order key (encode_for_index tag 0x07):

0x07 ++ srid(u32 BE) ++ morton_interleave(order-preserving axis codes) ++ exact axis codes

Each axis’s f64 runs through the same order-preserving u64 transform the Float index uses; the per-axis codes are bit-interleaved MSB-first into a fixed-width big-endian string (16 bytes for 2-D, 24 for 3-D) so byte order equals Z-curve order. The SRID leads the key, so points of different CRS occupy disjoint ranges. The exact coordinates are appended so the point decodes back exactly. A 2-D point key is 37 bytes. This is the final on-disk format — it ships in increment 1 so there is never a migration when the seek arrives.

Accelerating radius / bbox queries: the point-index seek

Create a point index so radius and bounding-box queries seek the Morton curve instead of scanning the whole label:

CREATE POINT INDEX loc_idx FOR (n:City) ON (n.loc)

Once the index exists, a query of the form

MATCH (c:City) WHERE distance(c.loc, point({latitude: 48.85, longitude: 2.35})) < 5000
RETURN c

or

MATCH (c:City) WHERE point.withinBBox(c.loc, point({x: 0, y: 0}), point({x: 10, y: 10}))
RETURN c

runs as a bounded set of curve-range seeks plus an exact post-filter: the query region is decomposed into a few contiguous Morton ranges (a superset of the answer), each is seeked in the value index, and the exact distance() / withinBBox filter trims the false positives. Results are identical to the full scan — the index only changes speed. EXPLAIN shows entry.spatial_seek in the Physical: footer when the seek is used. Without a point index the query still runs correctly, just as a label scan (Neo4j’s declare-to-accelerate model). A radius seek is roughly an order of magnitude faster than the full scan once the label is large and the region is selective.

Geographic edge cases are handled: a radius that crosses the ±180° antimeridian is split into two boxes and unioned; one that reaches a pole widens to all longitudes (a correct over-approximation the post-filter trims); an antimeridian-wrapping withinBBox (lower-left longitude greater than upper-right) covers [ll.lon, 180] ∪ [-180, ur.lon].

The seek also composes with a (non-kNN) ORDER BY — e.g.

MATCH (c:City) WHERE distance(c.loc, point({latitude: 48.85, longitude: 2.35})) < 5000
RETURN c ORDER BY c.name

still uses entry.spatial_seek to produce the candidate set, then applies the ORDER BY c.name as an ordinary post-sort over just those matches (byte-identical to a full scan + sort). Only a kNN ORDER BY distance(...) LIMIT k is served by its own path (above).

Nearest-neighbour (kNN)

MATCH (c:City) RETURN c ORDER BY distance(c.loc, point({latitude: 48.85, longitude: 2.35})) LIMIT 10

With a point index on c.loc, this runs an expanding-ring search over the curve: it grows a search box until the k-th nearest is provably confirmed (no un-searched point can be closer), then orders just that candidate set — identical to the full sort, but without touching every row. Without an index it falls back to the exact full sort.

Polygons & geometry (geometry MVP)

Beyond points, patinaDB has a first-class Polygon value and a small set of areal predicates — a scoped geometry MVP, not full PostGIS.

Constructing a polygon

// Exterior ring from a list of points (auto-closed if the last ≠ first):
RETURN polygon([point({x: 0, y: 0}), point({x: 10, y: 0}),
                point({x: 10, y: 10}), point({x: 0, y: 10})]) AS square

// With holes — a list of rings, ring 0 = exterior, rings 1.. = holes:
RETURN polygon([
  [point({x: 0, y: 0}), point({x: 10, y: 0}), point({x: 10, y: 10}), point({x: 0, y: 10})],
  [point({x: 4, y: 4}), point({x: 6, y: 4}), point({x: 6, y: 6}), point({x: 4, y: 6})]
]) AS ring_with_hole

The CRS is inherited from the points (all points must share one CRS — a mixed-CRS set is an error). A ring needs ≥ 3 distinct vertices. A polygon can be stored as a node/relationship property (it round-trips through storage) but is not spatially indexed (see limitations).

Predicates

// Point-in-polygon (ray casting, correct for holes):
MATCH (p:Place) WHERE within(p.loc, $region) RETURN p
// contains() is the same predicate with arguments swapped:
RETURN polygon.contains($region, point({x: 5, y: 5}))     // → true/false
// Polygon–polygon intersection (overlap or touch):
RETURN intersects($regionA, $regionB)                     // → true/false

• within(point, polygon) / contains(polygon, point) — even-odd ray-casting point-in-polygon: inside the exterior ring and outside every hole. A point exactly on an edge/vertex is reported inside (a documented boundary convention). Also available as the namespaced polygon.contains(polygon, point) / polygon.within(point, polygon).
• intersects(polyA, polyB) — a bounding-box reject fast path, then an edge-segment-crossing test, then a vertex-containment test (so containment with no crossing edges still counts). Returns true when the polygons overlap or touch.
• Mixed-CRS operands → null (parity with distance()).

Limitations (current increments)

• Geometry is a scoped MVP — a Polygon type with within / contains / intersects only. No linestrings, no multipolygon, no ST_* OGC function library, no spatial joins beyond the point predicates above.
• No polygon spatial index — polygon predicates are always a full scan + exact filter (a polygon is not added to the tag-0x07 point curve index; a polygon-column BVH / R-tree is an honest follow-on). A “points within a constant query polygon” optimization could later reuse the point bbox-seek.
• Polygons are 2-D — a polygon’s footprint is [x, y]; a 3-D point’s height is dropped on construction.
• wgs-84 polygons are treated as planar lon/lat — no antimeridian-crossing and no polar geometry (a polygon spanning the ±180° seam is out of MVP scope). Ray casting / intersection assume a flat plane, which is fine for local regions but not for large geodesic areas.
• intersects assumes well-formed input — degenerate or self-intersecting polygons are undefined (not asserted).
• distance() is spherical Haversine, not ellipsoidal (Neo4j’s own default; ~0.3% off the true geoid — fine for radius search, not survey work). Use point.distance.ellipsoidal() when you need the accurate WGS-84 geodesic.
• 3-D seeks are less selective than 2-D (interleaving three axes has worse curve locality), but still correct — the exact post-filter always runs.

Engrams

Every mutation in patinaDB is recorded. A engram is one committed unit of change — a list of low-level delta operations (create/delete vertex, set property, set/remove label, create/delete edge) plus metadata: an id, a parent id, a timestamp, and an optional message. The chain of engrams is the source of truth for history, diffs, time travel, and — in the server — replication.

Think of it as a git-like commit log for your graph: an append-only history you can inspect, diff, travel through, tag, and compact.

How writes become engrams

Autocommit (the common case). Every write — a REST /cypher call or a Bolt RUN — is captured and committed as one engram, atomically. A single statement, however complex (MATCH … CREATE … SET …, or a bulk UNWIND … CREATE), is one engram.

Explicit transactions. Over Bolt, BEGIN … COMMIT groups several statements into one engram applied atomically at COMMIT (see Bolt). ROLLBACK discards them — no engram is written.

Chunked bulk writes. CALL { … } IN TRANSACTIONS OF n ROWS commits one engram per chunk, so a large load is versioned in bounded steps instead of one giant one — see Procedures → bulk write clause.

Listing history

CALL patinadb.engrams() YIELD id, message, timestamp
RETURN id, message, timestamp ORDER BY timestamp DESC

Snapshots & compaction

To keep time travel fast, patinaDB periodically captures a full-graph snapshot (by default every 50 commits, configurable). Reconstructing a past state loads the nearest snapshot and replays deltas forward from there, rather than replaying the whole history. Snapshots are an internal optimisation — you interact with history through engrams, diffs, tags, and time travel.

Determinism & replication

A engram is a pure, deterministic description of a change: replaying a committed op stream reproduces the exact same graph, and a created vertex’s generated UUID is baked into the op so replays are stable. This is what lets the server replicate — a write becomes a Raft log entry of delta ops, and every node applies the same ops to reach the same state.

The engram lifecycle

History is a managed asset, not just an append-only ledger. These operations let you name, protect, compact, branch, and promote points in it.

Pin — protect an engram from compaction

A pinned engram is never coalesced by squash, so the point-in-time it marks stays reachable.

CALL patinadb.pin('<engram-id>')
CALL patinadb.unpin('<engram-id>')

Tag — a named, snapshotted, pinned point

A tag is a stable name for an engram (like a git tag), so you can refer to a meaningful point without tracking raw ids. Creating a tag pins its engram so squash cannot remove it. On a Raft server, the full snapshot is built asynchronously after the tag commits. Until then, reads reconstruct the tagged state through the existing history path; snapshot completion is an acceleration, not a prerequisite for correct reads. The embedded Dataset::create_tag API still builds the snapshot synchronously.

CREATE TAG v1                                  -- tag the current HEAD
CREATE TAG release AS OF '<engram-id>'      -- tag a specific engram
SHOW TAGS                                       -- list name → engram
SHOW JOBS                                       -- admin: local snapshot job status
DROP TAG v1                                      -- remove (unpins if unreferenced)

Read a database as it was at a tag with AS OF TAG. Tags replicate across a cluster — every node names and pins the same engram, then builds its snapshot independently. History-incomplete nodes retain the existing history-availability restrictions. See maintenance jobs for status, retries and the distinction between tag commit and snapshot readiness.

For a historical tag based on an existing snapshot, the background exporter retains one private primary-data base and replays each tag’s changes in a separate overlay. It omits query indexes from that base because snapshot records contain primary data and index definitions. Ordinary historical queries still use fully indexed reconstruction; the export base is never a query store. A cached ancestor may also serve tags beyond a newer snapshot when its complete replay chain can be verified. Reuse checks the history floor, unchanged compound index definitions, and snapshot-only transitions without a replayable delta. It declines optional reuse beyond 4096 engrams or 64 MiB of decoded serialized delta work, and retains the normal nearest-snapshot path. These are work estimates, not total-memory bounds. Each delta is still loaded and applied through the ordinary checked history path. Both temporary stores use the disposable-cache cap described under configuration. This avoids building those unused indexes, but still reads and writes snapshot data and can compete with live I/O. It is not a latency guarantee or a cap on total process memory. Snapshot publication keeps its existing durable completion boundary.

Squash — compact old history

Coalesce a run of old engrams into a single synthetic genesis, keeping recent history intact. Pinned (and too-recent) engrams are boundaries squash stops at. The live graph is unchanged; only the log is compacted.

CALL patinadb.squash(10)               -- keep the 10 most recent, coalesce older
CALL patinadb.squash(10, 1719792000)   -- …only those older than a unix timestamp

Fork — branch a database at a point

Create a new database seeded with the state of another at a chosen engram (HEAD if omitted). The fork starts with a single genesis engram and its own independent history. Forking is subject to the same user-database limit as CREATE DATABASE; reaching that limit rejects the fork before replication.

FORK DATABASE prod AS OF '<engram-id>' INTO staging

Restore — promote a past state to HEAD

Bring the state at a past engram back to the live graph as a new engram. History is preserved (nothing is rewritten); the restore is an ordinary append-only write, so it replicates cleanly.

CALL patinadb.restore('<engram-id>')

This is the write-side counterpart to time travel (which is read-only): time travel reads the past, restore promotes it to the present.

Large restores are chunked. The restore delta (the creates needed to reach the target state, then the deletes for everything not in it) is proposed in batches of PATINADB_RESTORE_OPS_CHUNK operations — default 5000, and the same knob governs POST /mgmt/restore. A restore smaller than one chunk is a single atomic entry, exactly as before. A larger one commits as several entries, which has one consequence worth knowing:

A chunked restore is not atomic. Intermediate, partially-restored states are observable between chunks, and a failure part-way through leaves the graph in one of them. Restore is an offline administrative operation — do not run it against a database serving live traffic and expect a clean cut-over. All creates are ordered before all deletes, so the intermediate states are supersets of the target, never missing data the target has.

Subscribing to changes (CDC)

The engram log is also a live change source: the server’s GET /changes endpoint streams each committed engram as it applies, resumable from an engram cursor — so external systems can react to graph changes as they happen (cache invalidation, search-index sync, ETL).

In the cluster

All of the lifecycle operations are replicated control commands: every node re-derives the result from its own identical history (synthetic genesis ids are content-derived, so the ids agree on every node). SHOW TAGS / SHOW DATABASES are local reads; the mutating commands go through the leader and need admin.

Maintenance jobs

For large portable backups use the explicit streamed archive and offline admin restore. Legacy JSON remains available; the new binary format avoids exporting all reconstructible history snapshot caches.

Long-running work and squash

Background execution does not remove the need for resource limits or query deadlines. Interactive queries remain synchronous and bounded; large maintenance needs explicit admission, status and completion semantics. Losing a connection is not proof that an admitted job was cancelled.

Prepared squash jobs are experimental and disabled by default. The explicit REST interface described below separates Raft admission, replica preparation, publication and completion. It does not change CALL patinadb.squash: that command remains synchronous, including its existing apply work and possible request timeout. Planning shares the maintenance artifact lane and streams the canonical identity, but this is not a production or performance qualification. A database with at most 4096 live vertices and edges plans on a separate small-database lane instead, so it does not wait behind a long tag, backup or squash job for a larger database. At most one such small squash runs beside one heavy job per node; heavy jobs remain serialized. The bound counts live records, not file size: a database file keeps its high-water allocation after history churn. The bound is a heuristic, not a measured memory limit.

The #1261 1M maintenance run6 still failed at the synchronous squash HTTP timeout. Post-squash PITR export/restore was not reached. Successful historical-read tests or component preparation tests do not supersede that failure; a new complete maintenance measurement is required. A subsequent #1261 component run on the 1M-ticket copy passed (222.090 s preparation, 5.145 ms local publication, 2.096 GiB maximum RSS; fresh process with warmed OS cache). Raw evidence: async-squash-1261-20260913-component-run2/summary.json and benchmark.log; repository report: docs/async-squash-1261-2026-09-13.md. A small real cluster also passes history-inclusive backup and pristine PITR restore after squash. Neither result substitutes for the complete mixed-load 1M maintenance gate. The repository qualification driver selects this experimental path with history_qualify.py --maintenance-only --async-squash; it requires committed completion before PITR export and checks pinned history on a pristine restore.

Experimental prepared squash

Keep PATINADB_EXPERIMENTAL_SQUASH_JOBS disabled until the deployment’s qualification gates pass. For an explicit experiment, enable it consistently on every participating node, including learners. All members must support protocol 16; do not downgrade a cluster containing durable squash-job records.

The endpoints require global Admin and access to the target database:

POST /mgmt/squashes
GET  /mgmt/squashes/<job_id>
POST /mgmt/squashes/<job_id>/cancel

Submit JSON on the leader, for example:

{"database":"default","keep_recent":10,"request_id":"baf2012c-205f-41b7-8bf5-612c10518826"}

request_id is an optional client-generated UUID. Reusing it with identical database/options returns the existing replicated record while it remains; different options conflict. Omitting it creates a new request. Optional before is a Unix-seconds cutoff. keep_recent must be at least one. A job handles one selected prefix of at most 4096 Engrams: completion does not guarantee that the whole database has reached the requested retention target.

HTTP 202 acknowledges durable admission, not completion. The record identifies the fixed prefix; ordinary graph appends do not move that job’s cutoff. One job per database can be active, with limits of 32 active jobs and 256 retained records cluster-wide. These are admission limits, not throughput measurements. Terminal-record deletion/expiry is not exposed yet; the retained-record ceiling can refuse additional submissions. Dropping a database retains terminal records: unfinished work becomes Superseded, successful work becomes Retired. This preserves request deduplication and the protocol-16 minimum across drop/recreate; dropping databases is not a way to reset that compatibility requirement or cap.

These are logical state names; the detail endpoint returns the serialized record. Preparation and publication receipts are local to each node. Both readiness barriers include every current member, including non-voting learners, not just a voter quorum. An unavailable learner can therefore delay completion. Membership changes require receipts from the new current membership.

Normal HEAD reads and graph writes can proceed during preparation; CPU, disk, MVCC retention and the artifact lane remain shared. During committed publication, affected history queries and retention metadata are fenced rather than exposing different old/new history boundaries. History backup is refused until publication is safe; retry after completion. Busy publication retries; neither preparation nor publication has a qualified latency guarantee.

Cancel explicitly while the record is Queued, including while a worker is preparing. A racing commit can win: cancellation then returns a conflict and recovery must finish the decision. Disconnecting a client never cancels a job. Do not use feature-flag changes, node shutdown or manual staging-file deletion as cancellation mechanisms.

Job decisions survive restart and ride Raft snapshots. A recovering node must verify its published boundary or recover the exact decided artifact; absence of a local staging manifest is not proof of cancellation. A node without the needed history remains fenced until recovery. Successful logical publication does not itself prove reclaimed disk, an off-host backup, or a successful PITR restore. The complete crash, learner, mixed-load and post-squash restore gates remain part of experimental qualification.

Disabling new admission does not stop recovery/finalization of committed jobs. Existing status/cancellation endpoints remain available; query job details on the leader for a fresh status barrier. Durable squash records automatically keep history backfill available for recovery, even without its experimental environment setting after restart. Cleanup may run after publication, but its progress is not a replicated job-completion guarantee and staging-artifact expiry is not provided.

Tag snapshot jobs

The Raft server runs tag-snapshot preparation outside serialized Raft apply. CREATE TAG returns after its name, exact Engram and retention pin are durably committed; it does not wait for the full snapshot. The tag is immediately usable through existing historical reconstruction where that history is available.

Waiting tag jobs retain their durable history reference, but do not hold a native database read version (#1485). A worker acquires its immutable view when preparation is admitted. If the tag is no longer at that view’s HEAD, the worker uses the retained historical reconstruction path. This can require more replay than capturing at submission; no preparation-latency guarantee is made. An active snapshot or query can still retain old disk pages while writes continue. Releasing a native reader permits reuse of eligible pages; it does not promise an immediate decrease in the database file’s allocated disk size. The durable history pin remains until the tag is removed. No new setting is required. A causal bookmark covers the tag commit, not background snapshot completion.

Historical fallback can reuse one private primary-data snapshot base between jobs, without constructing query indexes on that base. Each target replays its own deltas in an isolated overlay. This export-only base is separate from the fully indexed stores used for historical queries and is invalidated with history. Reuse can cross newer snapshot boundaries only with a checked ancestry chain, unchanged snapshot definitions and replayable deltas. Missing optional history, a floor boundary, snapshot-only transitions or the bounded replay-work ceiling retain the nearest-snapshot rebuild. No tag is declared ready before publication. Snapshot I/O still competes with live traffic; there is no background latency or total-memory guarantee.

Inspect the queue

As an administrator, use Bolt or the existing Cypher HTTP endpoint:

SHOW JOBS;
USE default SHOW JOBS;

Tag rows are local to the receiving node and selected database; backup rows are node-local full-registry jobs (database = '*') included in every database’s listing. The result is not a cluster-wide view; a remote learner may be at a different stage. HTTP returns the rows under jobs. Bolt returns these columns:

For tag rows, succeeded means this node has published the target snapshot. It does not mean all replicas have done so, nor that an off-host backup exists. There is no percentage estimate. SHOW JOBS currently has no WHERE, YIELD, ORDER BY, manual retry, cancellation or priority syntax. It requires global Admin.

Squash rows show the receiving node’s applied replicated state for the selected database, not a linearizable cluster-wide status poll. Their lowercase states are queued, committed, succeeded, cancelled and superseded; retired records also appear as succeeded here. Use the detail endpoint to distinguish retirement. Squash attempts and sanitized errors describe only the receiving node; waiting members and percentage estimates are not exposed. Cancel through the explicit REST endpoint, not SHOW JOBS syntax.

Recovery and resource behavior

Live tags are the persistent work list. Multiple tags pointing to one Engram share one job. Pending work is rediscovered after restart or history backfill; preparation can repeat, but publication is checked and idempotent. Failures retry with bounded backoff. Error/attempt history is a bounded in-memory diagnostic, not an audit log. Rows disappear when their last tag reference disappears.

One tag job executes per node at a time and shares the existing process-wide snapshot-build admission limit. A waiting job does not retain a read snapshot. An active job retains its immutable view and history dependencies. Normal graph writes can continue; destructive history operations may wait for that lease. Disk, CPU and retained MVCC pages are still shared resources, not isolated quotas. Historical targets may require on-disk reconstruction before snapshot streaming.

DROP TAG removes demand; moving a tag changes its target. An already-running serializer may finish its pass, but obsolete work is discarded before publication. Database replacement and snapshot installation also invalidate old prepared work. Do not explicitly unpin live tag targets whose history you need: the existing administrative unpin override still removes their retention protection. Dropping the final tag retains the usual unpin behavior.

This first increment covers server-side CREATE TAG. Embedded synchronous tagging and tagging inside existing bulk/upsert APIs retain their previous behavior. User queries and the legacy synchronous squash command have not been moved into the tag queue; experimental squash jobs use their own replicated lifecycle. The existing synchronous backup endpoint keeps its completion contract. During a rolling upgrade an older node may still build tag snapshots synchronously.

Asynchronous backups

All backup-job endpoints require global Admin and use the same authentication as /mgmt/snapshot. Archives contain every database and the user directory, including password hashes: protect downloads and server storage accordingly.

Submit on the leader, using HTTP POST:

POST /mgmt/backups?history=false&request_id=<client-generated-UUID>
POST /mgmt/backups?history=true&request_id=<client-generated-UUID>

The response is HTTP 202 with a persistent job_id. Repeating the same request ID on the same node, while its record remains, returns the existing job; changing its history option returns 409. Omitting the ID creates a new job each time. IDs are not cluster-wide retry tokens: after a lost response, inspect the original node rather than blindly resubmitting on another leader. PITR/history still requires its license entitlement and complete local history. These checks are repeated when execution begins.

GET    /mgmt/backups/<job_id>
GET    /mgmt/backups/<job_id>/file
DELETE /mgmt/backups/<job_id>

SHOW JOBS exposes queued, running, ready, failed, and expired for backups. Backup rows have an empty engram and no tags: the archive’s heads map names the per-database capture points. The detail endpoint also provides the history option, byte length, creation/expiry timestamps (Unix seconds), attempts, and a sanitized error. Capture occurs when the worker executes, not at enqueue time; it takes a fresh leader barrier. Queued jobs hold no database read views.

ready means the complete archive has been flushed, synced and published locally, not that an external backup exists. Download the file, persist it outside the cluster, and verify it using a restore drill. Completed HTTP streaming updates the existing backup-success stamp; this still cannot prove client-side persistence. Interrupted downloads retain the archive and can be restarted from byte zero. There is no Range/resume support in this increment.

Jobs and files belong to their original node and are not Raft-replicated. On restart, queued jobs remain queued, ready archives remain downloadable, and interrupted running jobs become failed. They are not silently retried against a different data cut. Submit a new ID to retry a failed job. Leadership loss before capture can fail the job; no automatic cross-node failover is promised.

One backup/tag serialization job runs per node through shared admission. Up to four backup downloads can be active. A node keeps at most 16 job records; delete terminal jobs to free slots (active jobs cannot be deleted). Ready archives expire after 24 hours; queued jobs expire 24 hours after submission. Expired files are reclaimed by maintenance, while status records remain until deletion. These are configured limits, not performance measurements.

PATINADB_BACKUP_JOB_MAX_BYTES limits each asynchronous archive when explicitly set. The legacy JSON format defaults to 16 GiB; format=archive has no default total-byte cap and uses the offline admin restore tool described above. PATINADB_BACKUP_JOB_MIN_FREE_BYTES sets the filesystem free-space floor (default 1 GiB). Both accept positive integer byte counts. Serialization checks available space periodically and fails rather than publishing a partial archive; this is not an exclusive disk reservation against other processes. Budget MVCC retention, Raft snapshots, downloads and normal writes separately. The old synchronous JSON endpoint is not governed by these job-specific limits; explicitly choosing format=archive applies the disk reserve and any configured byte budget there too.

The JSON backup/restore format is unchanged. Missing referenced history snapshot bodies now fail export rather than silently produce an incomplete PITR archive (#1289). Backup capture also opens the referenced snapshot files under a short, nonblocking retention lease (#1291). The backup retains those handles through serialization: later pruning or atomic replacement cannot remove its captured bodies. The lease is released before serialization, rather than held across the export. Active retention can refuse capture; submit a new job after it finishes. Each captured snapshot consumes a file descriptor until the view is dropped; descriptor exhaustion or a missing body fails capture, never produces a partial successful archive. Unlinked bodies still consume disk while handles remain open. Avoid concurrent database/schema/RBAC changes during preparation: the existing consistent-cut guarantee covers graph, HEAD and history, not an atomic snapshot of every live schema/access-control definition. Ordinary graph updates are covered by the pinned views. Ready-file checks detect missing or size-mismatched files, not arbitrary same-length corruption; retain independent integrity checks and restore drills for the downloaded recovery copy.

The legacy synchronous endpoint also retains cleanup ownership during preparation and the handoff to its response stream (#1292). Cancelling the request does not leave an unowned completed archive behind once the blocking preparation finishes. This is distinct from async jobs: disconnecting a client does not cancel a durable job, and its ready artifact remains available until expiry or deletion.

Large backups and offline restore

The versioned portable archive is the streaming alternative to the legacy JSON backup. It carries the registry’s graph, schema and users; history mode also preserves its Engrams and tag targets. Reconstructible history snapshot caches are omitted, but indispensable history bases and import/restore-marker snapshots remain, including markers that have older history as a parent.

Archive version 2 also preserves single-property registrations, compound index definitions and names used by DROP INDEX name. Online archives capture these from the same pinned view as the graph. Restore rebuilds compound index entries before publishing the destination. Use the matching server/admin revision: older readers reject version 2 rather than silently dropping its metadata. The new reader still accepts version 1, but cannot reconstruct index declarations that those archives never recorded (#1327). Recreate missing declarations from your schema migrations, then take a new backup; do not assume a legacy archive is a complete schema backup.

Online backup job

Submit on the leader using your existing global Admin credentials:

POST /mgmt/backups?history=true&format=archive&request_id=<UUID>
GET  /mgmt/backups/<UUID>
GET  /mgmt/backups/<UUID>/file

HTTP 202 means admitted, not complete. Download only when the job is ready; save the binary response as backup.pdb. Reusing the UUID with another history mode or format conflicts. History backup requires the existing PITR entitlement. Job status, retention and deletion otherwise follow maintenance jobs.

When PATINADB_BACKUP_JOB_MAX_BYTES is unset, this archive format has no fixed total-size cap. Setting it imposes an explicit byte budget. The disk reserve PATINADB_BACKUP_JOB_MIN_FREE_BYTES still applies; running out of capacity is an error, never a successful partial backup. Legacy JSON behavior is unchanged.

Required admin binary

Build both tools from the same revision:

cargo build --release -p patinadb-raft --bins

For a stopped registry, an offline backup avoids network transfer entirely:

patinadb-admin backup --from /data/patina --output /backups/backup.pdb --history
patinadb-admin verify /backups/backup.pdb
patinadb-admin restore /backups/backup.pdb --into /data/patina-restored

The server image also includes patinadb-admin. With Docker, override the server entrypoint using --entrypoint /usr/local/bin/patinadb-admin, mount the archive and target parent directory, and grant the container user access to those paths. Stop the source server before an offline backup; never restore over its mounted registry.

The output archive and restore destination must not already exist. There is deliberately no --force: the tool must never overwrite a running or mistaken target. Restore needs an existing parent directory on Linux and enough space for staging, index rebuilds and its free-space reserve. It does not contact a server and is not limited by an HTTP/query timeout. Run it as the intended database OS user, or deliberately adjust ownership before starting that user’s server; archive files and staging are private.

Both backup and restore accept --min-free-bytes (default 1 GiB). Restore also accepts an optional --max-archive-bytes policy limit. These do not provision space; monitor allocated disk and leave additional headroom for rebuilding indexes. The full restore’s memory/index-build behavior still needs qualification at your scale. The chunk payload is bounded to 1 MiB and manifest to 64 MiB; there is no graph-sized JSON snapshot in the new archive path.

verify checks the manifest and complete chunk/checksum chain. It is not a substitute for a restore drill: restore also validates graph/history records and rebuilds indexes. Truncated, corrupt or budget-exceeding input never publishes its staging registry. A crash may leave a private .patinadb-restore-* sibling; do not start a server on an unfinished stage.

Archives record each source database’s incarnation and history generation when it has one. An offline restore stores that as the restored database’s restored_from lineage; the database itself gets a new incarnation from the leader once the fresh cluster runs protocol 17. Archives from older servers restore without lineage. An archive carrying lineage is rejected by older patinadb-admin binaries (unknown field) rather than restored partially.

After restore, start a fresh cluster using the new registry directory and your normal license/configuration. Old Raft membership, logs, causal bookmarks and deduplication receipts are not restored. Do not drop this directory under an existing cluster identity. Users/password hashes and grants do survive: protect the archive like the original database. Checksums are not encryption or proof that the archive came from a trusted party.

Restored cluster bootstrap (#1325, development correction): older servers can let a new learner reach the causal log index without receiving the restored graph. Use the corrected server for the restored registry’s first start and initialization, not only when adding replicas later. A registry already initialized by an older server without the new bootstrap marker must be restored into a fresh registry for this procedure. Upgrading an already inconsistent cluster is not an automatic repair.

Initialize the restored registry with one local voter first (--bootstrap or a single-member POST /mgmt/init). The corrected server rejects direct multi-voter initialization of restored state. Before admitting the first new learner it durably checkpoints that state and retires the incomplete log prefix. An interrupted checkpoint remains pending across restart. A retryable DatabaseUnavailable response means admission has not completed: retry the same membership intent, rather than clearing bookmarks or manually copying live database files. The checkpoint can outlive an individual request. Its required baseline frontier is pinned durably on the first attempt: later writes remain a replayable tail rather than moving the retry target. A checkpoint from an obsolete causal domain is rebuilt before admission can complete. The server also waits for Raft’s snapshot-publication acknowledgement before requesting the corresponding log purge; a file appearing on disk is not enough.

Add the other nodes as learners, verify their graph and indexes, then promote the two additional local voters together. Keep regional replicas non-voting. Switch backend endpoints and trust configuration only after verifying the new cluster and imported grants. Old causal bookmarks are invalid in the fresh cluster. Do not silently discard them and retry a request; explicitly start a new recovery session after reconciling the acknowledged backup cut. Write-token receipts are also not restored: replaying an old write token can execute its mutation again. Reconcile uncertain pre-disaster writes before resubmission; tokens generated in the new cluster retain normal deduplication.

The development correction passed same-host SATA recovery at 64 and 4,096 tickets into three voters plus one regional learner, with three Rust/neo4rs backends repointed and fresh causal writes verified (#1261/#1325). Four real process-abort/restart boundaries passed at 64 tickets. Historical-cut checks cover the initially restored node, not immediate history availability on every replica. These are not independent-host, power-loss, WAN or 1M multi-node recovery results; keep deployment-specific restore drills.

Offline backup captures a stopped node’s durable local state; it does not prove that this node was current with its former leader. Recover incomplete installs and finish required history/publication recovery before taking a PITR backup.

Compatibility and qualification

format=archive is explicit. Existing JSON clients and POST /mgmt/restore remain unchanged; send the binary archive to patinadb-admin, not the JSON restore endpoint. The older embedded patinadb backup/restore format is also distinct. Unsupported magic/version is rejected rather than guessed.

Legacy JSON POST /mgmt/restore?force=true can create _restore_pre_* tags for the pre-restore state of existing databases. Treat those names as opaque. The #1323 development fix binds new names to their exact Engram instead of seconds, preventing repeated force restores from moving an earlier generated marker. Existing tags are not rewritten; this does not change archive restore or make force-merge restore a pristine disaster-recovery operation.

The previous #1261 JSON PITR run exceeded its configured 32-GiB budget and did not reach restore. That failed run remains failed. A new archive/admin drill on 2026-09-23 restored a one-million-ticket PITR archive under the current workflow: 3,896,529,031 transfer bytes, 259.682 seconds offline restore and 1,483,657,216 bytes sampled restore peak RSS (#1261). Boot-to-ready afterward was 0.202 seconds; do not confuse that with full recovery time. Mixed read p90s were 78–169 ms; several exceeded three times idle despite passing the existing 300-ms floor. No latency threshold was relaxed for this run.

A separate two-hour local five-node run recovered four HEAD/PITR archives into fresh isolated registries and checked graph, indexes, grants, constraints and subsequent writes (#1261). The source database directories were inaccessible during those restores. The large run selected PITR restore only; a HEAD export served as a contention control. These are single-host results, not off-host DR or a guarantee for your history size. See the qualification report for exact binary identity, failures, resource reserves and remaining gates.

Historical diff jobs

Use synchronous CALL patinadb.diffRange(from, to) for an interactive comparison. It still returns rows and respects query timeouts. For a larger comparison, an administrator can explicitly submit a node-local background job instead of holding a request open. This does not make arbitrary Cypher asynchronous.

Submit on the private management endpoint, using your operator authentication:

POST /mgmt/diffs
Content-Type: application/json

{"database":"default","from":"<old-engram-uuid>","to":"<new-engram-uuid>","request_id":"<client-generated-uuid>"}

Use JSON null for from to compare against an empty graph. Resolve tag names with SHOW TAGS first; endpoints are fixed Engram UUIDs, not moving names. HTTP 202 returns job_id and state. Retrying the same request ID and options on the same node returns the existing record; different options conflict.

Wait for ready before reading results. cancelling means work has not yet stopped; wait for cancelled before deleting. Other states are queued, running, failed and expired. Detailed failure diagnostics are in the node log. A failed job is not a partial successful diff.

Pages return rows and next; pass next as the subsequent offset until it is null. A row has change and data: change is vertices_added, vertices_removed, vertices_moved, vertices_changed, edges_added or edges_removed. Data is the corresponding core SnapshotDiff record; property values are typed JSON, e.g. {"Integer":42}. This format is not the synchronous procedure’s flat row schema. The existing comparison contract does not report secondary-label changes or edge-property-only changes.

Operational limits

Small forward ranges of property/secondary-label updates can compare only the affected existing vertices when the complete logged transition is provable. Existing snapshot point indexes are reused; without an index, a baseline scan still occurs but only selected vertices are retained. Structural mutations, snapshot-only transitions and unsupported ranges retain full comparison. This optimization is not the AS OF query engine and does not change its paths.

All operations require global Admin plus database authorization. Do not expose them as raw exports to application users with only label-level permissions. Jobs/results belong to the submitting node; polling another node is not failover. Queued and running jobs fail on restart rather than silently changing their source. Completed results survive restart, expire after 24 hours, and are not a backup or replicated audit record. Resubmit after losing the node.

Persisted state names remain queued, running, ready, failed, expired, cancelled and cancelling. An unrecognized state in the local job manifest is reported as an error; it is not silently shown as a failed job or rewritten. The manifest and result files are retained for diagnosis.

There are at most 16 retained jobs and one executing diff per node. Delete terminal entries when the queue is full. Diff execution does not take the tag/backup execution slot, but it still competes for host resources. Active comparison protects required history from destructive retention; queued jobs do not pin history and may fail if their endpoints are pruned before admission.

The default PATINADB_DIFF_MAX_BYTES is 4294967296 bytes of cumulative allocation accounting, not a hard RSS limit. It covers both endpoints and charges replay work without refunding deletions. Larger graphs or long update chains may need an operator-selected larger value and adequate host memory. This setting also applies to synchronous structural diffs. The engine compares snapshot records and checked deltas without constructing two temporary graph databases, but still holds comparison maps in RAM; there is no spill-to-disk fallback.

Jobs have a one-hour execution deadline. Results plus index are capped at 1 GiB, each encoded row at 1 MiB, and artifact writes check a 1 GiB free-space reserve. Pages contain at most 100 rows / 4 MiB encoded row payload; four page readers may be admitted concurrently. Cancellation is cooperative, not an immediate kill. Resource limits and damaged/missing history produce errors, not incomplete data.

The #1294 local 1M qualification covers the unindexed bounded update path, the structural fallback, near/distant endpoints, concurrent traffic, cancellation and restart using an explicitly larger accounting budget, not the default. Unindexed scans still take time; structural fallbacks still need full comparison maps. Queueing alone is not the bounded update optimization. See docs/history-diff-1294-2026-09-12.md in the source tree for measured intervals, resource use and failed attempts. No production approval follows from this single-host qualification.

History epochs (RFC-0038)

A history epoch is a contiguous section of one database history. It is written once, as a verified, immutable archive object, to a volume the operator provides. After an explicit, cluster-wide hot release, the members remove the epoch’s delta and snapshot bodies from local (“hot”) storage. AS OF reads, diff and tag exports of released history are then served from the archive.

Everything is opt-in. Without a policy or an operator request nothing is cut, nothing is released and nothing is deleted.

Requirements

• Every cluster member runs a release with peer protocol 18. Sealing is refused until all members are confirmed at 18. Once the cluster holds any epoch record, a new member must also speak 18.
• The database has an incarnation. A database gets one automatically once the cluster is confirmed at protocol 17.
• PATINADB_EPOCH_ARCHIVE_DIR is set to the same absolute directory path on every member, and every member sees the same shared volume. patinadb-raft --check-config rejects a relative path. The value is node-local; it never enters Raft.

A shared volume is not a backup. Losing it loses the archived copy, and after a release that copy is the only one. Back the volume up yourself, or take self-contained portable backups (see Backups and restore).

Sealing an epoch

An operator cuts an epoch:

curl -X POST http://leader:7474/mgmt/epochs \
  -H 'content-type: application/json' \
  -d '{"database": "sales", "upper": "<engram id>", "request_id": "<optional uuid>"}'

• upper must be an engram on the database’s current history chain.
• The first epoch starts at the chain root. Every later epoch starts exactly where the previous epoch ended. The server chooses that lower boundary.
• The leader validates the whole section against its own history and refuses if it cannot vouch for it, for example because of a history floor or fence.
• The call requires the admin role and access to the database, and is recorded in the audit log.

While an epoch is sealed, squashes of that database are refused, because a squash would rewrite the history being archived:

• CALL patinadb.squash fails with a non-retryable conflict.
• Prepared squash jobs are refused.
• Licence- or retention-driven squashes are deferred and logged.

Ordinary writes continue.

Automatic cuts (policy)

A per-database policy can cut epochs automatically. It is off by default and has no shipped thresholds:

curl -X PUT http://leader:7474/mgmt/epochs/policy \
  -H 'content-type: application/json' \
  -d '{"database": "sales", "max_engrams": 100000, "max_age_days": 90, "release_after_days": 7}'

• max_engrams cuts once that many engrams follow the previous epoch.
• max_age_days cuts at the newest engram at least that old.
• With both set, the newer boundary wins.
• release_after_days releases a committed epoch automatically after that grace period. Without it, release stays a manual step.

Every field is optional; a request without any field clears the policy. GET /mgmt/epochs/policy?database=sales shows it. The leader evaluates policies periodically and only while no seal is open. Engram timestamps only decide when to cut; the boundaries come from the ancestry. If HEAD’s timestamp lies more than five minutes in the future, or timestamps inside the candidate section run backwards, the age trigger is skipped and GET /mgmt/epochs/status reports clock_insane. The count trigger still applies.

Lifecycle

The leader exports a sealed epoch in the background, on the same lane as other maintenance artifacts, publishes the object and then commits the digests through Raft.

GET /mgmt/epochs?database=sales lists the catalog. On the serving node it also shows local progress of export, release barrier and local release: the attempts, the last error and whether work is running. Failed attempts are retried with increasing backoff, up to 10 minutes.

POST /mgmt/epochs/{seal_id}/verify checks the archive on the serving node. It returns verified, missing, corrupt, not_configured or not_committed. The result applies to that node only.

Hot release

curl -X POST http://leader:7474/mgmt/epochs/<seal_id>/release

Release is strictly all-member:

1. The leader asks every current member, learners included, for a receipt. Each member verifies the archive object completely (length, SHA-256, structure and identity) before it answers.
2. Only with a fresh positive receipt from every member does the leader propose the release. Apply re-checks the membership and that no tag or pin was created in between. A member that is offline holds the release up; there is no quorum shortcut.
3. Each member applies the release locally, after verifying the archive itself. It keeps a hot snapshot at the epoch’s upper boundary, indexes the range as released, then removes the range’s delta and snapshot bodies. Each step is durable before the next, so after a crash the node holds either the old hot state or a state the next attempt completes.

Releases go strictly in chain order; the released part is always a prefix of the history.

The release blocks (release_blocked, archive and hot data kept) when:

• a member reports that it cannot read the object (missing, wrong digest, wrong identity);
• a tag or pin lies inside the range (tags stay hot);
• receipts are missing for longer than the block period.

A blocked epoch needs a new POST …/release, or with release_after_days a new automatic attempt after another block period.

After a release:

• AS OF, patinadb.diff, snapshot diff and tag exports of released engrams hydrate a temporary, fully indexed store from the verified archive segment. At most two hydrations run per process; requests keep their deadline.
• Reads newer than the released range use the hot snapshot at its upper boundary and never touch the archive.
• A missing or unreadable volume fails the read with HISTORY_ARCHIVE_UNAVAILABLE (retryable). A damaged or mismatched object fails with HISTORY_ARCHIVE_INTEGRITY. A read is never answered from HEAD or with an empty result instead.
• CREATE TAG (also on HEAD when HEAD is the released boundary) and pins into released history are refused. Apply decides this against the replicated release, so a tag committed after the release is refused even before members have removed their local bodies; a tag committed first refuses the release instead.
• A fork whose fork point lies in released history is refused.
• CALL patinadb.squash and prepared squash jobs are refused for that database. A retention squash whose cutoff lies above the released range still runs.
• A retention cutoff inside the released range raises the history’s visibility floor instead of squashing. Reads below it fail with HISTORY_EXPIRED, even though the archive still holds them.

New members

Once anything is released, a new member is admitted only after it returns positive readiness receipts for every released epoch it may have to serve: those of each database’s current history version that retention has not expired entirely. A learner that joins from a physical snapshot backfills the released range’s metadata from the archive, not from peers, and the install restores the retention floor, so expired targets answer HISTORY_EXPIRED on the new member as well. For fully expired epochs the leader publishes a metadata-only artifact and commits its hash and length through Raft (protocol 20 required). A new member verifies this artifact before completing its chain, even after the full archive was collected. This does not restore expired query results or synthesize missing deltas (#1428).

For archives collected by older releases, a node with complete retained metadata can still publish the artifact. If neither the full archive nor an authoritative metadata source remains, backfill stays incomplete and retries; no missing ancestry is invented.

Archive collection

Nothing in the archive is deleted unless PATINADB_EPOCH_GC is set. The collector runs on the leader, at most once an hour, and removes only:

• staging files older than the grace period;
• objects no catalog entry references (leftovers of cancelled or superseded seals, duplicates), older than the grace period;
• with PATINADB_EPOCH_GC_BELOW_RETENTION also set: archives that lie entirely below their history’s retention floor, only after their replicated metadata artifact has been verified locally. Missing or corrupt metadata prevents full archive deletion. The metadata artifact remains referenced for the lifetime of its catalog record; orphaned metadata follows the ordinary grace period (#1428).

It never removes an object that a catalog entry of any incarnation (including an imported reference), an archive hold or a running export refers to. Licence loss or expiry never deletes anything. PATINADB_EPOCH_GC_GRACE_SECS sets the grace period (default 604800, seven days).

GET /mgmt/epochs/status shows the configured volume, trigger evaluations and the last collection pass.

Backups and restore

A ?history=true portable backup is self-contained by default: it re-embeds every released epoch from its verified archive segment. It restores on a cluster without any archive volume, and the restored history is entirely hot. Taking it needs a readable archive; without one the backup fails with HISTORY_ARCHIVE_UNAVAILABLE rather than produce a shortened history. History below a retention floor is not exported; the oldest visible engram becomes the restored history’s root.

GET /mgmt/snapshot?history=true&references_only=true keeps released epochs as references instead. The backup carries their catalog records, the hot snapshots at their boundaries and the retention floor, and places an archive hold on the referenced objects. List holds with GET /mgmt/epochs/holds and remove one with DELETE /mgmt/epochs/holds/{hold_id} once the backup is no longer needed.

A restore of a references_only backup:

• needs every member at protocol 18;
• verifies every referenced object on the serving node first. It is refused before anything is written when PATINADB_EPOCH_ARCHIVE_DIR is not set or an object is missing or damaged (HISTORY_ARCHIVE_UNAVAILABLE in the message);
• imports the epochs into the restored database’s new incarnation as read-only references to the source objects. They are never collected and never overwritten by the restored database.

references_only applies to GET /mgmt/snapshot JSON backups only. format=archive backups and asynchronous backup jobs always re-embed. An offline patinadb-admin archive backup has no archive reader, so it fails for a database with released history; take that backup online.

Archive layout and offline verification

<root>/v1/<incarnation>/g<history generation>/<seal_id>/segment-<sha256>.bin
<root>/v1/<incarnation>/g<history generation>/<seal_id>/manifest-<sha256>.json
<root>/.staging/

Object names are their SHA-256 digests. An existing object is never overwritten. A second export of the same seal, for example after a leader change, accepts an existing file only if it verifies to exactly the same content.

Verify an object offline, without a server:

patinadb-admin epoch-verify /srv/epochs/v1/.../manifest-<sha256>.json --replay

The command checks the manifest digest against its file name, the body length and digest, the record format and the parent chain. --replay also rebuilds the newest state of the epoch in a scratch directory (--scratch, by default the system temporary directory) and reports the number of vertices and edges.

Limitations

See Limitations.

Diffs

patinaDB can show what changed — both for a single engram and between any two points in history.

Single-engram diff

A git-show-style view of one engram: what was created, deleted, and which properties changed (old → new). Repeated SETs on the same property are coalesced, no-op sets are dropped, and prior values are resolved by reconstructing the parent state.

CALL patinadb.diff('<engram-id>')

Range diff (structural)

A structural diff between two reconstructed states — not a replay of the operations between them, but a comparison of the actual graphs at from and to. Omit from (null) to diff against the empty graph.

CALL patinadb.diffRange('<from-id>', '<to-id>')
CALL patinadb.diffRange(null, '<to-id>')

Move pairing

A naive structural diff reports a node that changed identity as one removed and one added node. The range diff is move-aware: it pairs a removed and an added vertex that share a label and match on an identity property, reporting a single move instead of an add/remove pair.

The identity-property priority list defaults to qualified_name, fqn, name.

For a long-running comparison, administrators can explicitly submit a historical diff job, observe it in SHOW JOBS, and retrieve a paginated result. Synchronous diffRange does not automatically change into a job and still honors query deadlines. Both paths share the direct snapshot/delta comparison engine and its allocation-accounting budget.

If two candidates match ambiguously, they are left unpaired (reported as separate add/remove) rather than guessed.

Time Travel

Historical reads use recorded snapshots and forward changes, not guessed undo against the current graph. For a selective node-property filter followed by a plain WITH node and one typed relationship hop, reconstruction can retain only the candidate nodes and their needed neighbors (#1261). The original query still evaluates all filters and OPTIONAL MATCH behavior. Broad or unsupported shapes fall back to the existing reconstruction path; this is not a universal cold-read latency guarantee. See limitations for qualification status.

Because every change is recorded as an engram, patinaDB can answer read queries against the graph as it was at any past engram. The state is reconstructed (nearest snapshot + forward delta replay) into a temporary view, and your query runs against that view. The live graph is never modified.

USE … AS OF

Prefix a read with USE <db> AS OF '<engram-id>' to travel back in a specific database:

USE sales AS OF '<engram-id>'
MATCH (o:Order) RETURN count(o)

Over REST you can equivalently pass an at field in the request body; over Bolt the USE … AS OF prefix is parsed per query.

Semantics & constraints

• Reads only. Time travel reconstructs a read-only past view. You cannot write to the past or “restore” the database to an old state through time travel (that’s a different operation — full snapshot import/export).
• Consistent point-in-time. A time-travel query sees the entire graph as it was at that engram — a coherent snapshot, not a mix of old and new.
• Cost. Reconstruction is bounded by the distance from the nearest snapshot to the target engram. Frequent snapshots (see Engrams) keep this cheap; querying a point far from any snapshot replays more deltas. See Performance & tuning below.

An unknown engram id is an ERROR, not the empty graph (changed in issue #590). AS OF an id that is not in the database’s history — a typo’d or stale id, or one squashed away by retention (see Engrams) — fails with an actionable unknown engram … error, so your application can always tell “that point in history is gone” apart from “the query matched nothing”. (Only the low-level Dataset::reconstruct API keeps its empty-on-unknown behaviour; the user-facing AS OF / restore entry points validate the id first.)

Even so, if your application time-travels to ids it stores, prefer a tag — it is pinned against squash, so the point it names stays reachable and SHOW TAGS tells you whether it is still there.

Damaged history is an ERROR too — never a smaller graph (issue #1145). The engram history is a chain: each point is reconstructed from the nearest stored snapshot plus the deltas above it. If any of those pieces is missing — a lost chain link, a lost delta, a snapshot file that went away while its metadata still claims it — the read fails, naming the engram it could not resolve.

It did not used to. Each missing piece read as “nothing here”: a lost chain link looked like “no snapshot in this history” and replay restarted from an empty graph, a lost delta replayed as “that commit did nothing”. The answer was a graph that never existed, returned with no error at all — measured in one real case at 5 nodes where 40 were correct. Nothing distinguished it from a genuinely small result.

You should never see this: it means storage damage or partial data loss, not a normal condition. What matters is that you now see it as an error instead of as data. Take a backup restore seriously if it appears — and note that a live (non-AS OF) query is unaffected, since it reads the current graph directly rather than replaying history.

Performance & tuning

An AS OF read is served by one of several paths. All of them return the byte-identical result — the choice affects only speed, and every acceleration falls back to a full reconstruct when it can’t prove it applies. That is why these knobs are safe to flip in either direction on a live system.

• Full reconstruct (the floor). Materialises the whole graph as of the target engram. Always correct, cost grows with graph size.
• Warm-window reconstruct (always on). The nearest snapshot’s base state is cached and shared by every read in that window; only the in-window deltas are replayed on top. A second read in the same window is dramatically cheaper than the first.
• Label-scoped partial reconstruct (PATINADB_PARTIAL_TIMETRAVEL, default on). When a query’s labels and relationship types are statically bounded, only that subgraph is rebuilt instead of the whole graph. An unbounded shape — a bare MATCH (n), a variable-length hop, a procedure call — falls back automatically. Read-only, so a write-only workload pays nothing for it.
• Point-index lookup (PATINADB_INDEXED_SNAPSHOTS, opt-in). For a cold single-vertex lookup anchored on id(n) = '<uuid>', a per-snapshot .snapidx sidecar turns a full-graph rebuild into a point read.

Practical guidance. Leave PATINADB_PARTIAL_TIMETRAVEL on; if you ever suspect it, =0 restores the full reconstruct for every read — slower, identical results. Reach for PATINADB_INDEXED_SNAPSHOTS only when a known set of historical points is read repeatedly: the sidecar is built lazily and in the background, so the read that triggers a build is itself served by the scan path and only later reads benefit. Bound its disk with PATINADB_MAX_SNAPSHOT_INDEXES (default 16 sidecars). Neither knob touches the write path, so neither can slow a commit or a cluster’s apply loop.

A tag requests a snapshot at its engram. Once that snapshot is available, reconstruction can start there without replaying earlier deltas. Raft-server materialization is asynchronous: acknowledgement is not snapshot readiness, and a snapshot does not guarantee fast queries. See the unresolved 1M historical workload findings in limitations.

Tags — named, snapshotted points in history

A tag is a stable name for an engram (like a git tag), so you can time-travel to a meaningful point without tracking raw engram ids. Creating a tag also pins its engram and requests a snapshot:

• Pinned — a tagged engram is protected from squash: compaction never collapses it, so the point-in-time it names stays reachable.
• Snapshot preparation — on a Raft server, a background job builds the full snapshot. Until it completes, reads use existing checked reconstruction; incomplete history is reported rather than replaced by the current state.

CREATE TAG v1                    -- tag the current HEAD
CREATE TAG release AS OF '<engram-uuid>'   -- tag a specific engram
SHOW TAGS                        -- list name → engram
DROP TAG v1                      -- remove (unpins if no other tag references it)

Read a database as it was at a tag:

USE default AS OF TAG 'v1' MATCH (n) RETURN n

On a cluster, CREATE TAG / DROP TAG replicate through Raft — every node names and pins the same engram. Snapshot preparation finishes independently on each node. AS OF TAG works where the required history is available, including followers. SHOW TAGS is a local read; CREATE/DROP TAG need admin and go through the leader. The returned bookmark covers tag creation, not snapshot completion. Use admin-only SHOW JOBS on the target node to inspect preparation. Embedded Dataset::create_tag remains synchronous.

Anamnesis

Anamnesis is patinaDB’s provenance/lineage system.

patinaDB is versioned by construction — the engram log already records what changed and when. Anamnesis turns that stream into a queryable W3C PROV-style property graph: for every write, who did it (an agent), the commit that did it (an activity), and — at the label / type and property level — what kind of change it made and how much.

The projection lives in a separate companion database named <db>__anamnesis, isolated from your main graph so it never touches your graph’s trees or indexes (main-graph read performance is unaffected). Because it is an ordinary database, it is replicated and carried in Raft snapshots for free, and you query it with plain Cypher.

The model

Provenance is aggregated by label/type + property, not one node per touched vertex. A write that creates a million :Ticket rows projects a handful of provenance nodes (one per label and property touched), not a million — the concrete uuids are not duplicated into the PROV graph (see Drilling to concrete uuids).

In <db>__anamnesis, each write projects to:

where op ∈ {created, updated, deleted} and on ∈ {node, edge}, and the relationships:

• (:Activity)-[:WAS_ASSOCIATED_WITH]->(:Agent)
• (:Activity)-[:AFFECTED {count, op}]->(:NodeType | :EdgeType)
• (:Activity)-[:SET {count}]->(:Property)

Agent, NodeType, EdgeType and Property are upserted by a content-derived stable id, so repeated writes never duplicate them: a (:NodeType {label:'Ticket', op:'created'}) touchpoint is created once and reused by every activity that creates Tickets. Each commit produces exactly one new :Activity, with AFFECTED/SET edges from that activity carrying the per-label counts.

Why op is on the type node. patinaDB edges are keyed by their (outbound, type, inbound) triple with no independent edge id, so two AFFECTED edges from the same Activity to a shared per-label node would collide. The created / updated / deleted touchpoints are therefore distinct :NodeType nodes (one per (label, op)), and MATCH (:NodeType {label:'X'}) returns up to three of them.

How ops map to touchpoints:

• AFFECTED (nodes): a CREATE → (label, created); a DELETE → (label, deleted); a property/label set or removal on a pre-existing node → (label, updated). Property/label sets on a node created in the same commit fold into created (creating a node with properties is one act, not a create-then-update).
• AFFECTED (edges): the same, keyed by the relationship type.
• SET (properties): every value set — SetVertexProperty / SetEdgeProperty, including on freshly created entities — records a (label, key) :Property touch. This is the property-level tracking axis: “which activity last set Ticket.status?”

The agent is the authenticated RBAC user, or "anonymous" when authentication is disabled. The timestamp is the leader-stamped commit time carried in the Raft entry, so it is identical on every replica.

Opt in

Provenance is off by default (zero cost when off — the write path does no provenance work). Enable it per database:

CALL patinadb.anamnesis.enable()      -- turn it on for the current/USE'd db
CALL patinadb.anamnesis.disable()     -- turn it off (companion data is kept)

USE sales CALL patinadb.anamnesis.enable() targets sales. Enabling creates the sales__anamnesis companion (idempotent) and replicates to every node. Requires the global Admin role.

Querying

The companion is a normal database — the primary path is a plain USE. Query provenance edges in the outbound Activity → target direction:

USE mydb__anamnesis
MATCH (a:Activity)-[r:AFFECTED {op: 'created'}]->(nt:NodeType {label: 'Ticket'})
RETURN a.timestamp AS when, r.count AS how_many
ORDER BY a.timestamp DESC

“Which activities set Ticket.status, and how often?”

USE mydb__anamnesis
MATCH (a:Activity)-[s:SET]->(:Property {label: 'Ticket', key: 'status'}),
      (a)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
RETURN ag.name AS who, a.timestamp AS when, s.count AS n
ORDER BY a.timestamp DESC

Query direction matters. Reading an edge property (r.count, r.op) across an inbound <- traversal currently returns NULL — a general engine limitation, not specific to provenance. Always traverse out of the :Activity ((a)-[r:AFFECTED]->…) when you need the edge’s count/op.

A convenience read procedure returns the activities that touched a node-label (newest-first):

CALL patinadb.anamnesis('Ticket')
YIELD engram_id, timestamp, message, op, count, agent

(It is server-side sugar for the USE <db>__anamnesis MATCH … query above. The argument is a label, not a vertex uuid — provenance is label-aggregated.)

The main-database engram log also records the writer: CALL patinadb.engrams() YIELD id, author, timestamp shows who committed each engram.

Drilling to concrete uuids

The PROV graph is deliberately coarse: it tells you which labels and properties an activity touched and how many — not which uuids. To get the concrete vertices a commit changed, drill into the engram delta log with the Activity’s engram_id:

CALL patinadb.diff('<engram_id>')   -- the per-vertex added/removed/changed detail

So per-vertex blame (“who last wrote this node”) is a diff-scan over history (cost O(history)), not an O(1) lookup. A reverse uuid → activities index is a possible future add if that access pattern becomes hot.

Enrichment

Auto-projected provenance records the structure of a write (who / what-kind / how-many). Enrichment lets the client attach context to the write — the run/source/model of the activity — so a pipeline can say why and from where it wrote, not just that it wrote. This is activity-level enrichment: each context entry becomes a property on that commit’s :Activity node.

It costs nothing unless you use it. When Anamnesis is disabled for the target database, a non-empty context is DROPPED — not stored, not queryable, gone (issue #1071). This is silent by default, on purpose (existing deployments are unaffected unless they opt in to the signal below), but it used to be silent with NO way to notice — a client faithfully sending {"actor": "enduser-4711"} on every write could believe it had months of attribution, right up until the first incident revealed Anamnesis had never been turned on for that database (or was turned on for a DIFFERENT one — enablement is per-database, and a newly created database always starts disabled). Two things now make the drop observable instead of invisible:

• Response signal. A write whose provenance/tx_metadata map is non-empty gets a provenance_recorded: true|false field in its response (REST body; a write with no such map carries neither key, so a client not using the feature sees no change). false means it was dropped.
• One warning per database, per node. The FIRST time a node sees a non-empty context land on a database without Anamnesis enabled, it logs a warning naming the database — once, not once per write (this deployment profile is 99% writes carrying the same context shape, so “once per write” would be log noise, not a signal).

If silent loss is unacceptable for your deployment, set --provenance-required (PATINADB_PROVENANCE_REQUIRED, off by default): a write carrying a non-empty context to a database without Anamnesis enabled is then refused outright (409, before anything is proposed — the graph is untouched) instead of silently dropped. Scope, honestly: this flag is enforced on the REST autocommit write path only today; Bolt writes (autocommit and explicit BEGIN…COMMIT transactions) are not yet covered — threading the check through Bolt’s result-streaming state machine is a larger, separate change than this issue’s “minimal, cheap” scope justified in one pass. A write with NO context is never affected by this flag, on either transport.

Attaching a context

REST /cypher — add an optional provenance object (an arbitrary key→scalar map) alongside the query:

{
  "query": "CREATE (:Doc {path: 'src/main.rs'})",
  "provenance": {
    "agent":  "indexer-run-42",
    "source": "git://repo@abcd123",
    "model":  "text-embedding-3-large"
  }
}

Bolt — drivers send it as transaction metadata, the canonical Neo4j mechanism. patinaDB reads tx_metadata from the RUN extra (autocommit) and the BEGIN extra (explicit transactions):

# autocommit
session.run("CREATE (:Doc {path: $p})", p="src/main.rs",
            metadata={"agent": "indexer-run-42",
                      "source": "git://repo@abcd123",
                      "model": "text-embedding-3-large"})

# explicit transaction — metadata set once at begin, applies to the whole tx
tx = session.begin_transaction(metadata={"agent": "indexer-run-42",
                                         "source": "git://repo@abcd123"})
tx.run("CREATE (:Doc {path: $p})", p="a")
tx.run("CREATE (:Doc {path: $p})", p="b")
tx.commit()   # one Activity, enriched with the begin metadata

Reserved keys

Some keys are special and are consumed (never folded onto the :Activity):

• agent (or its alias actor) sets the :Agent name instead of becoming an Activity property — so the recorded agent can be a pipeline or tool (e.g. "indexer-run-42", "nightly-etl") rather than the authenticated RBAC user. When both agent and actor are given, agent wins. Without either, the agent stays the authenticated user (or "anonymous").
• confidence, source and derived_from are the provenance values. They attach at a client-chosen granularity — see Scope: where the values attach below. By default (scope: changeset) they become :Activity properties, which is cheap. source and derived_from may each be a single scalar or a list — a list is stored as the sorted distinct set of values supplied (a single value stays a plain scalar, unchanged); use this when a write legitimately draws on more than one source.
• fields is an optional map ({propertyName: confidence}) giving a per-property confidence override, independent of scope — see Per-field confidence below. It is consumed, never emitted as an :Activity property itself.
• scope selects that granularity (changeset | label | attribute | instance). It is consumed, never emitted.

Every other key becomes an ordinary :Activity property.

Querying enrichment

Enrichment props are ordinary :Activity properties, and the overriding agent is an ordinary :Agent:

USE mydb__anamnesis
MATCH (a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
WHERE a.model = 'text-embedding-3-large'
RETURN ag.name AS agent, a.source AS source, a.timestamp AS when
ORDER BY a.timestamp DESC

Rules and limits

• Determinism. The context is folded into the projection on the Raft leader and baked into the prov ops carried in the same Raft entry, so every node records a byte-identical enriched Activity.
• Coercion (best-effort — enrichment never fails the main write). Values are coerced to a stored scalar: strings/numbers/booleans are kept; null values are dropped; nested lists/maps are stringified to JSON. Strings are truncated to 4096 characters, and at most 32 keys are folded (excess dropped in sorted order). Malformed or oversize metadata is clamped, never rejected.
• Structural keys are protected. engram_id, timestamp and message are owned by the projector; a context key with one of those names is ignored (it cannot overwrite the built-in Activity fields).

Scope: where the values attach

The three provenance values (source / confidence / derived_from) attach at a granularity you choose with the reserved scope key. One write supplies one set of values, applied uniformly to every target at that scope. This lets you record provenance cheaply at the commit level by default, and pay the per-entity cost only when you ask for it.