Introduction
patinaDB is a lightweight graph database written in Rust. It stores a property graph — vertices (nodes) and directed edges (relationships), each carrying typed properties — on top of redb, an embedded ACID key-value store with native cross-tree atomic commits. You query it with a Cypher-like language, and every write is recorded as a engram so you can inspect history and travel back in time.
patinaDB runs in two shapes from the same core engine:
- Embedded — reach it from Python via the bindings, from the
patinadbcommand-line tool, or from an AI agent via the MCP server, and open a database directory directly. No server, no network, no separate process. - The server (
patinadb-raft) — a standalone node that replicates writes with the Raft consensus protocol, hosts multiple databases, and speaks both a JSON REST API and the native Bolt protocol used by Neo4j drivers and the Neo4j Browser. A single node with--bootstrapis a drop-in lightweight server; add peers to get automatic failover.
What makes patinaDB distinct
- Versioned by construction. Every mutation is committed as an
engram — a git-like commit for your graph. List the history,
view a git-style diff of any single change, diff two arbitrary
points in time, tag a
meaningful state, and run read queries against the past with
USE … AS OF. No audit table to bolt on — the history is the database. - Provenance, for free. Because the history is a first-class change stream,
patinaDB’s Anamnesis system auto-projects it into a queryable PROV
graph in a companion database — who / what / when, and optional source /
confidence /
derived_fromat commit, type, attribute, or per-entity granularity, enriched straight from Neo4j transaction metadata. Where Neo4j leaves you to assemble this from APOC triggers or CDC, here it ships in the engine. See Anamnesis. - Cypher-compatible. The engine passes ~95% of the openCypher Technology Compatibility Kit (TCK). See Cypher Support for exactly what is and isn’t covered.
- Neo4j-tooling-compatible. The server’s Bolt endpoint works with the official Neo4j drivers and the Neo4j Browser, so you can point existing tools straight at patinaDB.
- Search is built in. Full-text search with BM25 ranking and
vector/embedding search (an IVF ANN index, cosine &
euclidean) ship in the engine — with the same procedure syntax as Neo4j
(
db.index.fulltext.queryNodes,db.index.vector.queryNodes). No external search service, no separate vector database. - Single binary, scales down and up. A one-node cluster (quorum of 1) behaves like a plain embedded server with no consensus latency; add peers for automatic failover (High Availability). Snapshots and time-travel reconstruction stream in O(chunk) memory, so the graph isn’t bounded by RAM.
Who this manual is for
This is the user manual — how to install, query, operate, and integrate patinaDB. It documents every user-facing feature and, just as importantly, its limitations.
A note on scope
patinaDB deliberately implements a subset of Neo4j/Cypher. It is not a drop-in Neo4j replacement for every workload — it targets embedded use, versioned/auditable graphs, and small-to-medium replicated deployments. Where behaviour differs from Neo4j, this manual calls it out explicitly. Read the Limitations chapter before committing to a production workload.
Installation
patinaDB ships as a Docker image containing three ready-to-run binaries:
| Binary | What it is |
|---|---|
patinadb-raft | The server — Raft-replicated node with REST + Bolt (the image’s entrypoint). |
patinadb | The embedded CLI — query / import / export / backup / restore / algo / changes over a database directory. |
patinadb-browser | The offline graph browser + admin dashboard. |
You do not need Rust, a compiler, or any build tooling — pull the image (or
install the standalone binary your vendor provided) and run it. The image is
published for both linux/amd64 and linux/arm64 — the same tag resolves to
the right architecture on either platform.
Run the server
docker run -p 7687:7687 -p 21001:21001 \
-v patinadb-data:/data \
-e PATINADB_AUTH_PASSWORD=change-me \
patinadb/patinadb
This starts a single self-leading node (a “cluster” of one — no peers needed) and exposes:
- REST on port
21001—POST /cypher - Bolt on port
7687— for Neo4j drivers and the Neo4j Browser
/data is where the graph, engram history, and node state live — mount a
named volume or bind mount so data survives a container restart.
By default the server refuses to start with no password set, so you don’t accidentally run an open database — see Authentication & TLS for how to configure it properly. For a disposable local try-out only (never expose this):
docker run -p 7687:7687 -p 21001:21001 patinadb/patinadb --insecure-disable-auth
Query it once it’s up:
curl -s -u neo4j:change-me -X POST localhost:21001/cypher \
-H 'content-type: application/json' \
-d '{"query": "CREATE (n:Person {name: \"Ada\"}) RETURN n"}'
See Quick Start for a walkthrough, and
Configuration Reference for every server flag
(--id, --addr, --db, --bootstrap, --bolt-addr, and the rest).
Run the embedded CLI
The same image also carries patinadb, the CLI for working with an embedded
database directory — no server required:
docker run -v "$PWD/mygraph:/data" --entrypoint patinadb patinadb/patinadb \
/data query "CREATE (a:Person {name: 'Ada'}) RETURN a"
If the binary is installed directly on the host (extracted from the image, or provided by your vendor as a standalone executable), run it the same way without Docker:
patinadb ./mygraph query "MATCH (n:Person) RETURN n"
See Command-Line Interface for every subcommand.
Run the graph browser
docker run -v "$PWD/mygraph:/data" -p 4200:4200 --entrypoint patinadb-browser \
patinadb/patinadb /data
Then open http://localhost:4200. See
Graph Browser & Admin UI for file / server / Bolt modes.
Python bindings
Install the patinadb Python wheel your vendor provided (or from your
organization’s package index):
pip install patinadb-<version>-<platform>.whl
from patinadb import Database
db = Database.open("./mygraph")
db.query("CREATE (n:Person {name: 'Ada', age: 36})")
rows = db.query_rows("MATCH (n:Person) RETURN n.name AS name, n.age AS age")
See Language Bindings for the full Python, MCP, and browser surface.
Next steps
- Quick Start — a five-minute walkthrough of both the CLI and the server.
- High Availability — add peers for automatic failover.
- Authentication & TLS — lock the server down before exposing it.
- Editions & Limits — what Community mode includes, and how to apply a license.
Quick Start
Two five-minute paths: the embedded CLI, and the server.
Embedded (CLI)
The CLI opens a database directory directly — no server.
# Create data (the directory is created on first use)
patinadb ./demo query \
"CREATE (a:Person {name: 'Ada', born: 1815})-[:KNEW]->(b:Person {name: 'Charles'})"
# Read it back
patinadb ./demo query \
"MATCH (a:Person)-[:KNEW]->(b) RETURN a.name, b.name"
# As JSON
patinadb ./demo query "MATCH (n:Person) RETURN n" --json
# What changed, and when?
patinadb ./demo log # list engrams
patinadb ./demo diff <id> # git-style view of one change
See Command-Line Interface for every subcommand.
The server
Start a single self-leading node (quorum of 1 — no peers needed):
docker run -p 21001:21001 -p 7687:7687 -v "$PWD/serverdata:/data" \
patinadb/patinadb --auth-user neo4j --auth-password secret
(Or run the patinadb-raft binary directly the same way if you have it
installed on the host: patinadb-raft --id 1 --addr 127.0.0.1:21001 --db ./serverdata --bootstrap --auth-user neo4j --auth-password secret.)
This exposes:
- REST on
127.0.0.1:21001—POST /cypher - Bolt on
127.0.0.1:7687— for Neo4j drivers and the Browser
Query over REST:
curl -s -u neo4j:secret -X POST 127.0.0.1:21001/cypher \
-H 'content-type: application/json' \
-d '{"query": "CREATE (n:Person {name: \"Ada\"}) RETURN n"}'
Connect the Neo4j Browser (browser.neo4j.io) to bolt://localhost:7687
with user neo4j / password secret, and run Cypher interactively — including
graph visualisation of returned nodes and paths.
Connect an official driver (Python shown):
from neo4j import GraphDatabase
drv = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "secret"))
with drv.session() as s:
for rec in s.run("MATCH (n:Person) RETURN n.name AS name"):
print(rec["name"])
From here:
- Add nodes for High Availability.
- Host more than one graph with Multi-Database.
- Lock it down with Authentication & TLS.
Cookbook: From Zero to Graph Analytics
This chapter is a single end-to-end walk-through: load → enforce → query → analyze → export → integrate, ending with the versioning and provenance features that make patinaDB distinct. Every command here is real and runnable against the embedded CLI; the outputs shown were captured from a throwaway database.
Throughout we use the CLI binary as patinadb <db> <subcommand> (see
Installation for how to get it — the Docker image, or a
standalone binary). The first argument is always the database directory —
it is created on first use.
We’ll build a tiny social graph: Person nodes connected by FOLLOWS
relationships.
1. Load data
For a one-off graph you can just CREATE nodes with a query (see
Quick Start). For anything larger, use the offline bulk
importer — neo4j-admin-style node and relationship files.
Two CSV files. Node files carry :ID, :LABEL, and typed property columns
(age:int, since:int, or a bare name for strings). Relationship files carry
:START_ID / :END_ID / :TYPE:
# people.csv
:ID,name,age:int,city,:LABEL
u1,Ada,36,London,Person
u2,Grace,40,New York,Person
u3,Linus,29,Helsinki,Person
u4,Margaret,33,Boston,Person
# follows.csv
:START_ID,:END_ID,:TYPE,since:int
u1,u2,FOLLOWS,2019
u2,u3,FOLLOWS,2020
u3,u1,FOLLOWS,2021
u4,u1,FOLLOWS,2022
u1,u3,FOLLOWS,2023
Import them (all node files load first, then relationships — so :START_ID
references resolve):
patinadb demo.db import --nodes people.csv --rels follows.csv
imported 4 nodes (12 props), 5 edges (5 props) in 0.00s (1093 nodes/sec)
The importer also reads Parquet and Arrow IPC (.parquet / .arrow /
.feather / .ipc) — dispatched per file by extension, with property types
taken from the columnar schema — and streams in bounded --batch-size batches,
so it scales to graphs far larger than RAM. Import maps each :ID string to a
deterministic UUID (--id-type hash, the default); pass --id-type uuid (or use
an :ID(uuid) header) to treat the id cell as a literal UUID, which is what makes
an export → import round-trip reproduce the same graph. See the
CLI chapter for every flag.
Import bypasses versioning. The bulk loader writes straight into storage, skipping the engram/history layer for speed — the imported data is visible at HEAD but not in
--attime-travel history. Load throughimportfor the initial data set; use queries (below) for changes you want in the audit trail.
Loading online, in batches
Inside a running query (embedded or on the server) you can stream a CSV and commit it in chunks, so a large load never buffers the whole file into one transaction:
LOAD CSV WITH HEADERS FROM 'file:///data/accounts.csv' AS row
CALL {
WITH row
CREATE (:Account {id: toInteger(row.id), name: row.name})
} IN TRANSACTIONS OF 1000 ROWS
Each batch of 1000 rows commits as its own engram (memory stays O(batch), not
O(file)), and on the server each batch is one Raft entry. ON ERROR {CONTINUE|BREAK|FAIL} controls what happens to a failing batch.
2. Enforce schema
Add constraints so bad data is rejected at write time. patinaDB supports
IS UNIQUE, IS NOT NULL, and IS NODE KEY:
patinadb demo.db query \
"CREATE CONSTRAINT person_name FOR (p:Person) REQUIRE p.name IS UNIQUE"
status: 'Constraint created on :Person(name) IS UNIQUE'
Creating a constraint over already-populated data fails loudly if the data already violates it. Once in place, a violating write is rejected:
patinadb demo.db query "CREATE (p:Person {name: 'Ada'})"
error: Unique constraint violation on :Person(name): value already exists
SHOW CONSTRAINTS lists them. On the server, constraint DDL replicates to every
node and rides Raft snapshots. See the Cypher chapter for the full
DDL grammar.
3. Query
Ordinary Cypher — MATCH, WHERE, ORDER BY, aggregation. List people by age:
patinadb demo.db query \
"MATCH (p:Person) RETURN p.name AS name, p.age AS age ORDER BY p.age DESC"
Who has the most followers? A traversal plus a count(*) group-by:
patinadb demo.db query \
"MATCH (:Person)-[:FOLLOWS]->(t:Person)
RETURN t.name AS name, count(*) AS followers
ORDER BY followers DESC, name"
The --json flag emits a machine-readable form whose scalars block is the
clean column view:
"scalars": {
"name": ["Ada", "Linus", "Grace"],
"followers": [2, 2, 1]
}
patinaDB passes ~95% of the openCypher TCK; see Cypher Support for the exact coverage and the Limitations chapter for the gaps.
4. Analyze
Built-in, read-only graph algorithms run over an in-memory snapshot of the current graph and yield real nodes plus a per-node result. Rank the influential accounts with PageRank:
patinadb demo.db query \
"CALL patinadb.algo.pageRank('Person','FOLLOWS') YIELD node, score
RETURN node.name AS name, round(score*1000)/1000 AS score
ORDER BY score DESC"
"scalars": {
"name": ["Ada", "Linus", "Grace", "Margaret"],
"score": [0.387, 0.374, 0.202, 0.038]
}
patinadb.algo.wcc (weakly-connected components) and patinadb.algo.degree
(degree centrality) round out the set, with gds.*.stream aliases for tooling.
See Procedures.
To understand the shape of your data before writing queries, inspect the planner’s statistics catalog — per-property count, distinct values (NDV), and min/max:
patinadb demo.db query \
"CALL patinadb.stats('Person') YIELD label, property, count, ndv, min, max
RETURN property, count, ndv, min, max ORDER BY ndv"
"scalars": {
"property": ["age", "city", "name", null],
"count": [4, 4, 4, 4],
"ndv": [4, 4, 4, null],
"min": [29, "Boston", "Ada", null],
"max": [40, "New York", "Margaret", null]
}
The same catalog feeds cost-based entry-point selection and join ordering — it only ever changes which plan runs, never the result. See Procedures → Statistics catalog.
5. Export
The complement of import: scan the whole graph read-only into
import-compatible files — one node file per label, one relationship file per
type:
patinadb demo.db export --out dump
exported 4 nodes (1 files), 5 edges (1 files) to dump in 0.00s
$ ls dump
nodes_Person.csv rels_FOLLOWS.csv
$ cat dump/nodes_Person.csv
:ID(uuid),age:int,city,name,:LABEL
1ac09593-6a05-5203-b12a-91294189c587,40,New York,Grace,Person
1be3441a-1076-5b1a-97f5-da854d48b8f3,33,Boston,Margaret,Person
263ca574-2e85-505f-bb21-e4c490b055da,29,Helsinki,Linus,Person
440d7304-d691-50fa-806a-888ead31a245,36,London,Ada,Person
Because the id column is written under an :ID(uuid) header, re-importing
reproduces the same graph, same UUIDs — a lossless round-trip:
patinadb fresh.db import --nodes dump/nodes_Person.csv \
--rels dump/rels_FOLLOWS.csv --id-type uuid
# imported 4 nodes (12 props), 5 edges (5 props)
--format parquet / --format arrow write the columnar equivalents. From within
a query you can also export selectively with CALL patinadb.export.csv(label, path) or CALL patinadb.export.query(query, file) — see
Procedures → CSV export. On the server,
GET /mgmt/export?db=…&format=… streams a portable tar archive of a whole
database.
6. From code
Python
The Python bindings (PyO3 + maturin) give you a data-science-friendly, row-oriented API on top of the same embedded engine:
import patinadb
db = patinadb.Database.open("./demo.db")
# Row-oriented results: list[dict] keyed by RETURN column — pandas-ready.
rows = db.query_rows(
"MATCH (p:Person) RETURN p.name AS name, p.age AS age ORDER BY p.age DESC")
import pandas as pd
df = pd.DataFrame(rows) # columns: name, age
# Bulk-load DataFrame records straight into the graph (deterministic UUIDs
# from the id key, so re-runs are idempotent and edges reference by key):
db.bulk_load_nodes("Person", people_records, id_key="id")
db.bulk_load_edges("FOLLOWS", follows_records)
# Thin wrappers over the graph-algorithm procedures:
ranked = db.page_rank(label="Person", rel_type="FOLLOWS")
Install the patinadb Python wheel to use this (pip install patinadb-<version>-<platform>.whl). See Language Bindings.
AI agents (MCP) and the browser
- The MCP server (
patinadb-mcp) exposes a database to an AI agent as structured tools —cypher/write/schema, analytics (statistics,graph_algorithm,import_csv), and the versioning differentiators (engrams/diff/time_travel_query/provenance). See Language Bindings → MCP. - The Graph Browser & Admin UI opens a database directly (or connects to a server over REST/Bolt), visualizes the graph, paints PageRank / WCC / degree results onto it, and surfaces the cluster/metrics admin dashboard.
- The server speaks Bolt, so the official Neo4j drivers and the Neo4j Browser point straight at patinaDB.
7. Time-travel & provenance — the differentiators
Every write that goes through a query is recorded as an engram — a git-like commit. Build up some history:
patinadb social.db query "CREATE (a:Person {name:'Ada', team:'core'})"
patinadb social.db query "CREATE (g:Person {name:'Grace', team:'core'})"
patinadb social.db query "MATCH (a:Person {name:'Ada'}) SET a.team = 'infra'"
List the history (oldest first) with patinadb social.db log, then view a
git-style diff of any single change:
patinadb social.db diff <set-engram-id>
engram ae2eae1a-3fcc-5c26-aad9-b11c4949b5c3
parent: 54f17ad5-bc1d-5686-867a-9891bc906d92
timestamp: 1783448468
~ (7530d903:Person).team: 'core' -> 'infra'
Query the graph as it was at any past engram — no audit table required:
# Ada's team at the engram just before the SET:
patinadb social.db query \
"MATCH (a:Person {name:'Ada'}) RETURN a.team AS team" --at <prev-engram-id>
# team: 'core' (it is 'infra' at HEAD)
On the server the same reads use the USE db AS OF '<engram>' selector (or a
named TAG). See Time Travel, Diffs, and
Engrams.
Finally, Anamnesis auto-projects that change stream into a
queryable W3C-PROV graph in a companion <db>__anamnesis database — who, what,
when, and optional source / confidence / derived_from — enriched straight from
Neo4j transaction metadata. It is opt-in per database
(CALL patinadb.anamnesis.enable()) on the server, where writes flow through
Raft. Where other databases leave you to assemble provenance from triggers or
CDC, here it ships in the engine.
Where to go next
- Cypher Support — the full query language and its coverage.
- The Server (Raft) — run patinaDB as a replicated service.
- Full-Text & Vector Search — search in the engine.
- Limitations — read this before a production workload.
Data Model
patinaDB stores a labelled property graph.
Vertices (nodes)
A vertex has:
- A stable UUID identity (assigned on creation, baked into the engram log so replays are deterministic).
- A primary label plus zero or more secondary labels (multi-label nodes
are supported;
labels(n)returns all of them). - A set of properties — string keys mapped to typed values.
CREATE (a:Person:Employee {name: 'Ada', born: 1815})
Edges (relationships)
An edge is directed, has exactly one type (a label), connects two vertices, and may carry its own properties:
CREATE (a)-[:KNEW {since: 1833}]->(b)
Edges can be traversed in either direction, and patterns may be left-to-right,
right-to-left, or undirected (-[:KNEW]-), in which case both directions
are searched.
Property value types
The AttributeValue type covers:
| Type | Example literal | Notes |
|---|---|---|
| String | 'hello' | Order-preserving in the value index. |
| Integer | 42 | 64-bit signed. |
| Float | 3.14, 1.0e9, .5 | 64-bit IEEE-754. |
| Boolean | true, false | First-class (AttributeValue::Bool). |
| Null | null | Drives three-valued logic (see below). |
| List | [1, 2, 3] | Heterogeneous; indexable, sliceable. |
| Map | {a: 1, b: 'x'} | Nested. |
| Temporal | date('2026-06-28'), datetime(...) | Date / Time / LocalTime / DateTime / LocalDateTime / Duration. |
| Path | result of a path pattern | Sequence of nodes and edges. |
Three-valued (Kleene) logic
Comparisons and boolean operators follow SQL-style three-valued logic. Any
comparison involving null yields Unknown, not true or false:
RETURN 1 = null // null (Unknown)
RETURN null AND false // false (false dominates)
RETURN null OR true // true (true dominates)
IS NULL / IS NOT NULL remain strictly two-valued, by specification. In a
WHERE filter, Unknown collapses to “not matched”.
Storage internals (informational)
Properties are kept in a label-scoped, order-preserving value index, so a
scan over :Person.born never returns :Robot nodes with the same value, and
sorted pagination over a property is O(limit) rather than a full scan.
Compound indexes over several fields accelerate equality-prefix + sort
queries. You normally don’t manage these directly — they are maintained
automatically on writes. To declare your own indexes and enforce data integrity,
see Constraints and Query Planning;
Limitations lists the current DDL gaps.
Cypher Support
patinaDB implements a large, Neo4j-compatible subset of Cypher. The engine is validated against the openCypher Technology Compatibility Kit (TCK): the latest run passes ~3712 / 3868 scenarios (~95.9%). The TCK is treated as a prioritized gap report, not a strict regression gate — so treat this chapter as the authoritative statement of what works.
Reading
MATCHwith node/edge patterns, optional labels, property maps, and all three directions (->,<-,-undirected).OPTIONAL MATCH.- Variable-length paths —
-[:T*1..3]->,[*0..n](length-0 self-row),shortestPath. WHERE— full boolean expressions:AND/OR/NOT/XOR, comparisons,IN,IS NULL,STARTS WITH/ENDS WITH/CONTAINS, regex=~, nested parenthesisation.RETURNwithDISTINCT, aliases (AS), expressions, ordering.WITHchained query stages (projection, filtering, aggregation hand-off, ordering before projection).UNWIND,UNION/UNION ALL.ORDER BY/SKIP/LIMIT— single-key ordering uses an index fast-path; multi-key falls back to a post-sort.
Loading CSV
LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
WITH toInteger(row.id) AS id, row.name AS name
CREATE (:Person {id: id, name: name})
LOAD CSV [WITH HEADERS] FROM '<url>' AS <var> [FIELDTERMINATOR '<c>'] streams
rows from a CSV file into the query as a row source — like UNWIND, but from a
file. It is read row by row (the file is never fully buffered).
WITH HEADERS→ each row is a map keyed by the header, sorow.columnNameworks. Without it, each row is a list of string cells:row[0],row[1], ….- Cells are strings (openCypher semantics). Convert explicitly with
toInteger/toFloat/toBoolean. A coercion may sit directly inside aCREATE/MERGEpattern property —CREATE (… {p: toInteger(row.x)})— as well as in aWITH; only arow[i]list-index expression still needs aWITHstage first. FIELDTERMINATOR '<c>'overrides the delimiter (default,).- Sources:
file:///absolute/path,file://host/absolute/path(the host is ignored), and bare/relative filesystem paths. Quoted fields and embedded delimiters/newlines are handled.http(s)://URLs are not yet supported.
Large loads (memory): a plain
LOAD CSV … CREATEbuffers all its writes into one transaction (on the server, one Raft entry), so a very large load has an O(rows) memory cliff. Wrap it inCALL { … } IN TRANSACTIONS(below) to chunk it into many small commits — the online answer. For millions of rows the offlinepatinadb <db> importbulk-loader is faster still. ALOAD CSV … RETURN rowread query streams end-to-end.
Server-side file reads (security): on the server,
LOAD CSV FROM 'file://…'reads a server-side file, so it is deny-by-default: it reads only directories whitelisted with--allow-csv-dir, and any file-I/O query is raised to require the global Admin role. See Bulk Loading → Server file-I/O sandbox and Authentication & TLS. The embedded/CLI path installs no sandbox and is unaffected.
Batched writes: CALL { … } IN TRANSACTIONS
LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CALL {
WITH row
CREATE (:Person {id: toInteger(row.id), name: row.name})
} IN TRANSACTIONS OF 1000 ROWS
CALL { <write-subquery> } IN TRANSACTIONS [OF <n> ROW[S]] [ON ERROR {CONTINUE|BREAK|FAIL}] chunks a large write into many small, independently
committed transactions instead of one giant one. The outer stream (typically
LOAD CSV … AS row or UNWIND $rows AS r) feeds rows to the inner subquery,
which runs and commits every n rows as a separate transaction (default
n = 1000). This bounds peak memory to one chunk — the fix for the
LOAD CSV … CREATE RAM cliff — and, on a cluster, replicates as one small entry
per chunk instead of one unbounded one.
- Each chunk is its own commit ⇒ its own engram (so batched ingest is versioned and time-travellable, unlike the offline importer), and partial progress is durable: a crash between chunks recovers to a chunk boundary (all-or-nothing per chunk). A later chunk sees the writes committed by earlier chunks.
OF <n> ROW[S]sets the chunk size (omit for the 1000 default).ON ERROR—FAIL(default) aborts the whole statement on a failing chunk;CONTINUEskips the failing chunk and commits the rest;BREAKstops after the failing chunk, keeping the chunks already committed. A skipped/failed chunk changes nothing (it is resolved against a throwaway copy of the graph, so a partial chunk is never left behind).- The
CALL { … } IN TRANSACTIONSmust be the final clause of the query.
Writing
CREATE,MERGE(match-or-create),SET(properties, map merge, andSET n:Labellabel mutation),REMOVE,DELETE/DETACH DELETE.FOREACH.- Writes inside
WITHstages.
Expressions & functions
- Arithmetic, string, list, and map operators; list/map indexing and slicing.
- Aggregation —
count(incl.count(*)),sum,avg,min,max,collect, withGROUP BYsemantics (mix of aggregated and grouping keys inRETURN),DISTINCTinside aggregates, and aggregate hoisting through nested function calls and arithmetic. CASE(simple and generic).- List comprehensions and pattern comprehensions.
- Quantifiers —
any/all/none/single, including aggregate andcount(*)sources. - Scalar functions —
toInteger,toString,size,labels,type,nodes,relationships,range, and many more. - Temporal functions —
date,time,localtime,datetime,localdatetime,duration, plus namespaced calls (date.truncate,duration.between,datetime.fromepoch) with ISO-8601 week numbering.
Procedures
CALL invokes built-in procedures (history, diff, full-text search). See
Procedures. User-defined procedures are not supported.
DDL
-
CREATE INDEX/DROP INDEX/SHOW INDEXES— single-property and compound (multi-field) B-tree indexes. -
CREATE CONSTRAINT/DROP CONSTRAINT/SHOW CONSTRAINTS— uniqueness (IS UNIQUE), existence (IS NOT NULL), and node-key (IS NODE KEY). See Constraints. -
CREATE EDGE SORTED INDEX(patinaDB extension) — pre-orders, per anchor vertex, that anchor’s targets over one relationship type by a target property, so a traversal-fan-out +ORDER BY target.prop [DESC] LIMIT kis served by a seek + take (no fan-out, no post-sort — bench Q5–Q8). Two orientations:CREATE EDGE SORTED INDEX [name] FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at CREATE EDGE SORTED INDEX [name] FOR (m:Ticket)<-[:HAS_LABEL]-() ON m.created_at SHOW EDGE SORTED INDEXESCreating the index over a populated graph backfills existing edges; it is kept live on writes.
EXPLAINnameslimit.edge_sorted_topkonce a covering index serves the query. On a server it replicates to every node (deterministic re-run). Thenameis optional and cosmetic (the def is identified by its shape). Not yet carried in Raft snapshots — see Limitations. -
CREATE FULLTEXT INDEX/DROP INDEX/SHOW FULLTEXT INDEXES— see Full-Text Search. -
CREATE DATABASE/DROP DATABASE/SHOW DATABASES(server only) — see Multi-Database. -
EXPLAINandPROFILErender the physical operator tree.
Query introspection
EXPLAIN MATCH (p:Person)-[:KNEW]->(q) WHERE p.born < 1850 RETURN q
PROFILE MATCH (p:Person {name: 'Ada'}) RETURN p
EXPLAIN shows the chosen plan (scan strategy: compound index, property-value
index, label scan, or all-vertices) without running it; PROFILE runs and
annotates it.
What is not supported
A precise list lives in Limitations. The headline gaps:
no user-defined procedures, no stored procedures/triggers, and the ~5% of
TCK scenarios that remain — mostly exotic temporal/list edge cases. (Index DDL
and uniqueness / existence / node-key constraints are
supported; see Limitations for the exact scope.) Side-effect
counters (+nodes, -relationships in query summaries) are not tracked.
Bulk Loading & Import
patinaDB offers three ways to get a lot of data into a graph, from fastest and least featureful to fully online and versioned:
| Path | Where | Versioned? | Memory | Best for |
|---|---|---|---|---|
patinadb import (offline) | CLI, embedded | No — HEAD only | O(batch) streaming | Millions of rows into a fresh or offline database. |
LOAD CSV … CALL { … } IN TRANSACTIONS (online) | Cypher | Yes — one engram per chunk | O(chunk) | Live ingest into a running (possibly clustered) database. |
LOAD CSV … CREATE (online, un-chunked) | Cypher | Yes — one engram | O(rows) cliff | Small loads, or LOAD CSV … RETURN reads. |
Offline: patinadb import
The offline bulk-loader reads neo4j-admin-style CSV, Parquet, or Arrow IPC
files and writes them straight into the embedded store in durable batches,
bypassing Raft and the engram/versioning layer. That is what makes it fast —
and it means the loaded data is visible at HEAD but is not in --at
time-travel history. All node files load first, then all relationship files.
patinadb ./mygraph.db import --nodes people.csv --nodes companies.csv \
--rels works_at.csv
Real run (three node rows + two company rows + three edges):
imported 5 nodes (11 props), 3 edges (3 props) in 0.00s (1058 nodes/sec)
File format
Files follow the neo4j-admin convention. Node files carry a :ID column, a
:LABEL column, and typed property columns; relationship files carry
:START_ID / :END_ID / :TYPE plus property columns.
:ID,name,age:int,active:boolean,:LABEL
p1,Ada,36,true,Person
p2,Grace,40,true,Person
:START_ID,:END_ID,:TYPE,since:int
p1,c1,WORKS_AT,2018
- CSV property types come from a
name:typeheader suffix —:int,:float,:boolean,:date,:localdatetime, or a barenamefor string. - Parquet / Arrow property types come from the file’s Arrow schema
(
Int*/UInt*→ Integer,Float*→ Float,Utf8→ String,Boolean→ Bool,Date32/64+Timestamp→ temporal); an unsupported column type is a loud error, not a silent drop. A null cell means the property is absent. - Dispatch is by extension (
.csv/.parquet/.arrow/.feather/.ipc) or forced with--format.
Deterministic ids and idempotent re-runs
By default (--id-type hash) each :ID string is hashed to a deterministic
UUIDv5. This is what lets a relationship file reference endpoints by their
business key with no id-map, and it makes a re-run idempotent (the same
:ID always maps to the same vertex UUID).
With --id-type uuid (or an auto-detected :ID(uuid) header) the id cell is
the literal vertex UUID. This is the round-trip mode: an export writes
:ID(uuid), so re-importing reproduces the same graph, same UUIDs included.
patinadb ./mygraph.db export --out ./dump --format parquet
patinadb ./fresh.db import --nodes ./dump/nodes_Person.parquet \
--rels ./dump/rels_KNOWS.parquet
See the CLI reference for every flag (--batch-size,
--delimiter, --skip-ref-check, …) and the matching export
command. On the server, GET /mgmt/export streams the same
import-compatible files as a tar archive (see the REST API).
Throughput reality
Bulk load is value-index-write-bound, not parse-bound: every property write
also maintains the label-scoped value index. In practice this is roughly
25–38k nodes/sec on one box, and Parquet is not meaningfully faster than
CSV — the columnar reader is quicker, but the index writes dominate. Memory
stays O(batch-size) regardless of file size (streaming).
Online: LOAD CSV
LOAD CSV streams rows from a CSV file into a running query as a row source —
like UNWIND, but from a file. Unlike the offline importer, it goes through the
normal write path, so each load is versioned (an engram) and, on a cluster,
replicated.
LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CREATE (:Person {id: toInteger(row.id), name: row.name})
WITH HEADERSmakes each row a map keyed by header (row.name); without it each row is a list of string cells (row[0]).- Cells are strings — coerce with
toInteger/toFloat/toBoolean. A coercion may sit directly in aCREATE/MERGEpattern property (as above); arow[i]list-index expression still needs aWITHstage first. FIELDTERMINATOR '<c>'overrides the,delimiter.http(s)://URLs are rejected; onlyfile://and bare/relative paths are read.
The full clause reference is in Cypher Support → Loading CSV.
The un-chunked memory cliff
A bare LOAD CSV … CREATE streams the source row by row, but buffers all
resolved write operations into one transaction (on the server, one Raft
entry) before committing — an O(rows) memory cliff for a large load. A
LOAD CSV … RETURN row read query streams end to end and has no such cliff.
Online + chunked: CALL { … } IN TRANSACTIONS
Wrapping the write subquery in CALL { … } IN TRANSACTIONS chunks the load into
many small, independently-committed transactions — the online answer to the RAM
cliff. Peak memory drops to one chunk, and on a cluster each chunk replicates as
one small Raft entry instead of one unbounded one.
LOAD CSV WITH HEADERS FROM 'file:///data/more_people.csv' AS row
CALL {
WITH row
CREATE (:Reader {id: row.id, name: row.name, age: toInteger(row.age)})
} IN TRANSACTIONS OF 2 ROWS
Loading a 5-row file OF 2 ROWS produces ⌈5/2⌉ = 3 chunks — and three
separate engrams, one per committed chunk:
$ patinadb ./mygraph.db query "MATCH (r:Reader) RETURN count(r) AS n"
n: 5
$ patinadb ./mygraph.db log | wc -l
3
- Each chunk is its own commit ⇒ its own engram, so batched ingest is versioned and time-travellable (unlike the offline importer). A later chunk sees the writes committed by earlier chunks (read-your-writes).
OF <n> ROW[S]sets the chunk size (default1000).ON ERROR—FAIL(default) aborts the whole statement on a failing chunk;CONTINUEskips it and commits the rest;BREAKstops after it, keeping earlier chunks. A skipped/failed chunk changes nothing — it is resolved against a throwaway copy of the graph, so a partial chunk is never left behind.- Crash-atomic per chunk: a crash between chunks recovers to a chunk boundary. The graph and the engram log always agree.
- The
CALL { … } IN TRANSACTIONSmust be the final clause of the query, and it cannot run inside an explicit BoltBEGIN … COMMITtransaction.
On the server the driver proposes one client_write per chunk, so a
250-row load OF 100 ROWS advances the applied index by 3, not 1 — bounded Raft
entries, read-your-writes preserved across chunks on the leader.
Server file-I/O sandbox
Reading a file:// URL, or writing with the export procedures,
touches the server’s filesystem. On the server both are deny-by-default
and gated by two layers:
- Directory sandbox —
--allow-csv-dir <dir>(reads) and--allow-export-dir <dir>(writes) whitelist directories; unset means every file access is refused. Paths are canonicalized, so..traversal and symlink escapes are rejected. - Authorize by effect — any query that does file I/O is raised to require the global Admin role, because file access is a host-level capability, not a graph-data one. A per-database Writer cannot read or write host files.
The embedded library and the CLI install no sandbox, so they are unaffected (local file access is the point). See Authentication & TLS → Cypher-driven file I/O.
Which path should I use?
- A one-off migration into a fresh database, or millions of rows → offline
patinadb import(fastest; give up in-history versioning of the load). - Live ingest into a running / clustered database →
LOAD CSV … CALL { … } IN TRANSACTIONS OF n ROWS(versioned, replicated, bounded memory). - A small load, or you need the CSV as a read source → plain
LOAD CSV … CREATE/LOAD CSV … RETURN row.
Constraints
patinaDB supports uniqueness, existence (NOT NULL), node-key, and
property-type (IS :: <TYPE>) constraints on nodes, plus existence and
property-type constraints on relationships, with Neo4j-5 (and legacy
Neo4j-4) DDL. Constraints are enforced at write time, replicate across a
cluster, and are carried in backups and Raft snapshots.
Creating constraints
CREATE CONSTRAINT [name] [IF NOT EXISTS]
FOR (n:Label) REQUIRE n.prop IS UNIQUE
Node kinds:
| Kind | DDL | Enforces |
|---|---|---|
| Uniqueness | REQUIRE n.prop IS UNIQUE | No two :Label nodes share a non-null value for prop. |
| Existence | REQUIRE n.prop IS NOT NULL | Every :Label node has prop set (non-null). |
| Node key | REQUIRE (n.p1, n.p2) IS NODE KEY | The tuple is present on every :Label node and unique (composite unique + existence). |
| Property type | REQUIRE n.prop IS :: INTEGER | When present, prop’s value is of the declared type (does not require presence). |
Relationship kinds (FOR ()-[r:TYPE]-(), any direction):
| Kind | DDL | Enforces |
|---|---|---|
| Existence | REQUIRE r.prop IS NOT NULL | Every :TYPE relationship has prop set (non-null). |
| Property type | REQUIRE r.prop IS :: INTEGER | When present, prop’s value is of the declared type. |
The legacy Neo4j-4 forms are also accepted:
CREATE CONSTRAINT … ON (n:Label) ASSERT n.prop IS UNIQUE and the shorthand
ON :Label(prop). The property-type predicate also accepts the IS TYPED <TYPE>
synonym of IS :: <TYPE>.
Property types
IS :: <TYPE> (and IS TYPED <TYPE>) accepts: BOOLEAN, STRING, INTEGER,
FLOAT, POINT, DATE, LOCAL TIME, ZONED TIME, LOCAL DATETIME,
ZONED DATETIME, DURATION, and LIST (BOOL/INT are accepted as aliases).
A missing or null property is allowed — a type constraint restricts a
present value’s type only; combine it with an existence constraint if you also
need the property present. Element-type refinement (LIST<INTEGER NOT NULL>) is
not yet supported (bare LIST only).
$ patinadb ./cdb.db query \
"CREATE CONSTRAINT person_email FOR (p:Person) REQUIRE p.email IS UNIQUE"
status: 'Constraint created on :Person(email) IS UNIQUE'
$ patinadb ./cdb.db query \
"CREATE CONSTRAINT emp_key FOR (e:Employee) REQUIRE (e.dept, e.num) IS NODE KEY"
status: 'Constraint created on :Employee(dept, num) IS NODE KEY'
The optional name identifies the constraint for DROP and SHOW. IF NOT EXISTS makes creation idempotent.
Creating over existing data
CREATE CONSTRAINT first validates the current graph. If the existing data
already violates the constraint (a duplicate value, or a missing required
property), the statement fails and no constraint is registered — fix the data and
re-run.
Enforcement semantics
Enforcement runs at write time on CREATE, MERGE, and SET / REMOVE:
$ patinadb ./cdb.db query "CREATE (:Person {email: 'ada@x.io', name: 'Ada'})"
$ patinadb ./cdb.db query "CREATE (:Person {email: 'ada@x.io', name: 'Ada2'})"
error: Unique constraint violation on :Person(email): value already exists
- UNIQUE is NULL-exempt. A null (or absent) value is not constrained —
many nodes may omit
prop. Only two concrete equal values collide. (Use a node-key or a separate existence constraint if you also need the property present.) - Existence and node-key reject a missing/null required property — including
a
CREATE/MERGEthat omits it, aSET n.prop = null, or aREMOVE n.prop. This holds for both node existence and relationship existence (checked at edge create,SET r.prop = null, andREMOVE r.prop). - Property-type rejects a present wrong-typed value on
CREATE/MERGE/SET(node or relationship). A missing/null value is allowed. - The uniqueness check is an
O(log N + matches)seek of the label-scoped value index, so it is cheap even on a large label. - Node-key uniqueness is served by an automatically registered backing compound index over the key tuple — you do not create it separately.
Inspecting and dropping
$ patinadb ./cdb.db query "SHOW CONSTRAINTS"
constraints: ['person_email: :Person(email) IS UNIQUE', 'emp_key: :Employee(dept, num) IS NODE KEY', 'person_name: :Person(name) IS NOT NULL']
DROP CONSTRAINT person_email [IF EXISTS]
Replication and durability
- Cluster replication is automatic. Constraint DDL replicates through the
Raft log and re-runs deterministically on every node (the same
IndexDdlpath that index DDL uses). - Raft snapshots carry the defs. A follower that bootstraps purely from a streamed snapshot (after the DDL log is purged) still enforces every constraint.
- Portable backups carry the defs too.
GET /mgmt/snapshot/POST /mgmt/restoreround-trip UNIQUE, existence, and node-key defs (alongside vector indexes, tags, and RBAC users) — restore re-issues them asCREATE CONSTRAINT … IF NOT EXISTS. See the REST API.
Not supported
- Relationship key / uniqueness (
FOR ()-[r:TYPE]-() REQUIRE r.id IS UNIQUE/IS KEY) — rejected with a clear “not supported yet” error (needs relationship-property indexes). LIST<T>element-type refinement — only bareLISTis accepted.- Bare composite UNIQUE without existence — use
IS NODE KEYfor a multi-property key (which also requires presence).
See Limitations for the full schema-feature scope, and the Data Model for how properties and the value index work.
Edge-Sorted Indexes
An edge-sorted index keeps, per anchor vertex, that anchor’s neighbours over one relationship type pre-ordered by a property of the neighbour. It turns the query shape “a node’s neighbours, ordered by a property of the neighbour, top-N, paginated” from a traverse-then-sort into a seek + scan — no fan-out, no post-sort.
This is a patinaDB extension. The grammar reference lives in Cypher Support → DDL; this chapter is the conceptual and worked-example companion.
The problem it solves
One shape shows up nearly everywhere you build on a graph:
- a user’s most recent tickets,
- the newest comments under a post,
- a person’s activity feed / timeline,
- the top-N items per parent, ordered by a timestamp, score, or price.
In Cypher it reads:
MATCH (u:User {name: 'Ada'})-[:REPORTED]->(t:Ticket)
RETURN t
ORDER BY t.created_at DESC
LIMIT 10
Simple to write, but normally expensive. To return ten rows the engine must:
- traverse the whole fan-out — every
REPORTEDedge Ada has, even if she filed 50 000 tickets, - read the sort property (
created_at) off every one of those targets, then - sort all of them just to discard everything below the top ten.
The work is proportional to the anchor’s degree, not to the LIMIT. For a
hub vertex — a power user, a popular post, a busy parent — that is the difference
between a feed that renders instantly and one that stalls.
What patinaDB does
An edge-sorted index stores a sorted adjacency: for each anchor, its targets over one relationship type are already laid out in target-property order. The query above becomes:
- seek to the anchor’s slice of the index,
- iterate in
created_atorder (DESCwalks it in reverse), - take
kand stop.
That is O(n + k) (skip n, take k) — no fan-out, no materialising the
whole neighbour set, no post-sort. A power user with 50 000 tickets costs the
same top-ten as one with twelve. The index is maintained synchronously on
every write (edge create/delete, and moving a target when its sort property
changes), so it is never stale.
The DDL and a worked example
The syntax names the traversal shape and the target property to order by. Both orientations are supported:
-- outbound anchor: (anchor)-[:REL]->(target)
CREATE EDGE SORTED INDEX [name] FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at
-- inbound anchor: (target)<-[:REL]-(anchor)
CREATE EDGE SORTED INDEX [name] FOR (m:Ticket)<-[:HAS_LABEL]-() ON m.created_at
SHOW EDGE SORTED INDEXES
The name is optional and cosmetic — a def is identified by its shape
(direction, relationship type, target label, target property), so two CREATEs
of the same shape are idempotent.
The following was run end-to-end against the CLI on a tiny User → REPORTED → Ticket graph; the output is real.
Build the graph
CREATE (u:User {name:'Ada'})
CREATE (t1:Ticket {id:1, title:'Login fails', created_at:'2026-07-01T09:00:00'})
CREATE (t2:Ticket {id:2, title:'Slow dashboard', created_at:'2026-07-03T14:30:00'})
CREATE (t3:Ticket {id:3, title:'Export broken', created_at:'2026-07-05T08:15:00'})
CREATE (t4:Ticket {id:4, title:'Typo in header', created_at:'2026-07-06T11:45:00'})
CREATE (u)-[:REPORTED]->(t1)
CREATE (u)-[:REPORTED]->(t2)
CREATE (u)-[:REPORTED]->(t3)
CREATE (u)-[:REPORTED]->(t4)
Create the index
$ patinadb feeddb query \
"CREATE EDGE SORTED INDEX ticket_feed FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at"
status: 'Edge sorted index ticket_feed created FOR ()-[:REPORTED]->(:Ticket) ON created_at'
$ patinadb feeddb query "SHOW EDGE SORTED INDEXES"
edgeSortedIndexes: ['FOR ()-[:REPORTED]->(:Ticket) ON created_at']
Creating the index over a populated graph backfills the existing edges, so you can create it any time.
Page 1 — newest first
$ patinadb feeddb query \
"MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
RETURN t.id, t.title, t.created_at
ORDER BY t.created_at DESC LIMIT 2"
t.created_at: ['2026-07-06T11:45:00', '2026-07-05T08:15:00']
t.id: [4, 3]
t.title: ['Typo in header', 'Export broken']
Confirm the index serves it
EXPLAIN shows the physical plan. When an edge-sorted index covers the query,
the Physical: footer names limit.edge_sorted_topk:
$ patinadb feeddb query \
"EXPLAIN MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
RETURN t ORDER BY t.created_at DESC LIMIT 2"
plan: 'Limit skip=0 count=2
Project items=1 distinct=false return_star=false
Sort keys=1
MatchScan required=[(u:User) -[entry](t:Ticket) → PropertyIndex[created_at] SORTED SCAN DESC est.5]
Physical:
limit.edge_sorted_topk
'
If you drop the index (or the query shape does not match), the footer names a
different path — always run EXPLAIN to confirm the index is doing the work.
Page 2 — keyset pagination
To page forward, carry a cursor: the sort value of the last row on the
previous page (2026-07-05T08:15:00), and ask for rows strictly beyond it. For
a DESC feed that is <:
$ patinadb feeddb query \
"MATCH (u:User {name:'Ada'})-[:REPORTED]->(t:Ticket)
WHERE t.created_at < '2026-07-05T08:15:00'
RETURN t.id, t.title, t.created_at
ORDER BY t.created_at DESC LIMIT 2"
t.created_at: ['2026-07-03T14:30:00', '2026-07-01T09:00:00']
t.id: [2, 1]
t.title: ['Slow dashboard', 'Login fails']
Keyset pagination is stable under concurrent writes (unlike SKIP n, which
shifts when rows are inserted) and never re-scans the pages you have already
seen.
Honest note on the cursor page. The bare top-N shape (page 1) is served by
limit.edge_sorted_topk. Adding the keysetWHERE t.created_at < …currently changes the plan shape, so page 2 falls back tolimit.bounded_topk— a bounded top-K heap that is O(k) memory and returns the correct rows, but still reads the fan-out to apply the filter. Pushing the cursor into an edge-sorted-index seek (so deep pages are O(log n + k)) is a planned follow-up. In practice the first page — the one users actually load — is the hot path, and it is fully covered.
When to use it — and when not
Reach for it when you have the “top-N neighbours by a neighbour property” shape over a skewed fan-out (some anchors have far more neighbours than others) and you want the first page to stay fast regardless of degree. Feeds, timelines, “latest N per parent”, and leaderboards-per-group are the sweet spot.
Each index covers exactly one (direction, relationship type, target label, target property) combination. If you sort the same neighbours by two different
properties, or traverse two relationship types, you create one index per shape.
Create it after a bulk load. Bulk import does not maintain edge-sorted
indexes incrementally; CREATE EDGE SORTED INDEX after the load backfills the
whole graph in one pass, and every write from then on keeps it live.
Skip it when the fan-out is small and uniform (a plain traverse-then-sort is
already cheap), when you never LIMIT the result (you want all neighbours in
order — there is no top-N to accelerate), or when the target property changes
very frequently on high-degree anchors (see the write-cost note below).
Advisor: discovering you need one
You do not have to guess whether a query would benefit. patinaDB ships an
advisor that recognises the traversal-fan-out + ORDER BY target.prop LIMIT
shape and, when it is running the slow fan-out top-K only because no covering
edge-sorted index exists, tells you the exact statement to create. It is
advice only — it never creates anything and never changes how a query runs.
In EXPLAIN / PROFILE. When the plan falls onto the fan-out
limit.bounded_topk for a coverable shape, the plan text carries an Advice:
line naming the index that would flip it to limit.edge_sorted_topk:
EXPLAIN MATCH (u:User {uid: 1})-[:REPORTED]->(t:Ticket)
RETURN t ORDER BY t.created_at DESC LIMIT 20
…
Physical:
limit.bounded_topk
Advice: CREATE EDGE SORTED INDEX FOR ()-[:REPORTED]->(m:Ticket) ON m.created_at (would serve this ORDER BY … LIMIT via edge_sorted_topk instead of a fan-out top-K)
Run that CREATE EDGE SORTED INDEX and the advice disappears — the plan now
reads limit.edge_sorted_topk. That round-trip is the guarantee: an advice is
emitted iff creating the named index would actually engage the fast path (it
reuses the executor’s own coverage decision), so there are no false positives.
As a procedure. CALL patinadb.advisor(query) analyses an arbitrary query
string and returns one row per suggestion:
CALL patinadb.advisor(
'MATCH (u:User {uid: 1})-[:REPORTED]->(t:Ticket) RETURN t ORDER BY t.created_at DESC LIMIT 20'
) YIELD suggestion, reason, current_plan
| column | meaning |
|---|---|
suggestion | the exact CREATE EDGE SORTED INDEX … statement to run |
reason | why it is suggested (the shape it matched, the plan it runs today) |
current_plan | the plan the query uses right now (limit.bounded_topk) |
It returns no rows (no error) when there is nothing to suggest: a
non-traversal query, a shape an edge-sorted index cannot serve (variable-length,
undirected, multi-anchor, no ORDER BY … LIMIT, a filtered/unlabeled target), or
a query already served by a covering index. Use it to sweep your hot read
queries and discover the exact indexes to declare.
Honest positioning and tradeoffs
How this compares to other graph databases. General-purpose graph databases,
Neo4j included, index a relationship’s own properties, or a node’s
properties — not a per-anchor adjacency pre-sorted by a neighbour’s property.
For the top-N-neighbours-by-neighbour-property shape that means the usual plan is
traverse-then-sort, exactly the cost this index removes. The underlying idea of
a sorted adjacency is not new — bespoke feed and social-graph stores have long
kept time-sorted association lists for precisely this access pattern — but
exposing it as a declarative, general-purpose index you can CREATE over any
(rel, target-property) pair, inside a Cypher database, is uncommon. That is the
differentiator, stated plainly.
Write cost. The index is maintained synchronously, and the cost is not free:
- Creating or deleting an edge updates one index entry — cheap.
- Changing a target’s sort property moves that target in every anchor’s slice
that points at it. For a target with high in-degree (many anchors point
to it), a single
SET t.created_at = …is O(in-degree) index moves. If your workload rewrites the sort property often on well-connected targets, weigh that against the read speedup.
Snapshot-carry caveat (clustered mode). In a Raft cluster the index
definitions replicate via the Raft log — every node re-runs the DDL and
builds its own copy, deterministically. They are not yet carried in Raft
snapshots. A node that bootstraps purely from a streamed snapshot (after the
log that carried the CREATE was purged) will lack the def and quietly fall back
to traverse-then-sort until the DDL is re-issued. Correctness is never
affected — only speed — and the fallback path returns identical rows. Re-issue
the CREATE EDGE SORTED INDEX statements after such a bootstrap (they are
idempotent) to restore the fast path. This is tracked in
Limitations.
Query Planning & Performance
patinaDB compiles each query into a streaming operator tree (the “Op tree”), picks a physical access path for every entry point, and — on skewed data — uses a statistics catalog and a cost model to choose between competing plans. This chapter shows how to read what the planner chose and which fast paths make a query cheap.
Reading EXPLAIN and PROFILE
EXPLAIN renders the chosen plan without running the query; PROFILE runs
it and prefixes the elapsed time.
$ patinadb ./mygraph.db query "EXPLAIN MATCH (p:Person {name: 'Ada'}) RETURN p"
plan: 'Project items=1 distinct=false return_star=false
MatchScan required=[[entry](p:Person) → PropertyIndex[name] = est.1]
'
Read the tree bottom-up: the MatchScan is the leaf that produces rows, and each
line above consumes them. The arrow annotation is the access path and its
estimated cardinality — here a PropertyIndex[name] = est.1 point lookup on the
value index (one estimated row), not a full label scan.
A range predicate with an ORDER BY that a value index can serve turns into a
sorted seek and emits a Physical: footer naming the executed access path:
$ patinadb ./mygraph.db query \
"EXPLAIN MATCH (r:Reader) WHERE r.age > 40 RETURN r.name ORDER BY r.age"
plan: 'Project items=1 distinct=false return_star=false
MatchScan required=[[entry](r:Reader) → PropertyIndex[age] SORTED SCAN ASC est.2]
Physical:
entry.range_seek
'
The Physical: footer is the single source of truth for the executed access
path — the same registry drives both the EXPLAIN render and the executor, so
EXPLAIN cannot lie about which path runs. Common footer names:
| Footer | Meaning |
|---|---|
entry.range_seek | Value-index range seek (aligned >/>=/</<= bound). |
entry.cost_override: <rule> (≈N rows) beats <priority-rule> on cost | The cost model overrode the default priority pick (see below). |
limit.bounded_topk | ORDER BY … LIMIT k via an O(k)-memory top-K heap (no full sort). |
limit.edge_sorted_topk | Traversal + ORDER BY target.prop LIMIT k served by an edge-sorted index. |
The statistics catalog
The planner keeps a lazy, cached statistics catalog derived from the value
index — per label a vertex count, and per property the present count,
distinct-value count (NDV), min/max, and a small equi-depth histogram. Inspect it
with CALL patinadb.stats:
$ patinadb ./mygraph.db query "CALL patinadb.stats('Person')"
For the imported Person data (3 nodes) this yields, per property:
| property | count | ndv | min | max |
|---|---|---|---|---|
| (label) | 3 | — | — | — |
active | 3 | 2 | false | true |
age | 3 | 3 | 32 | 40 |
name | 3 | 3 | Ada | Grace |
The catalog is invalidated automatically by writes (it is tagged with each label’s write generation), computed on demand, cached, and — because statistics only change which plan runs, never the result — never replicated across a cluster. Each node computes its own.
Cost-based selection
Two things use the catalog:
-
Entry-point selection. For a skewed EQ predicate (a value with many rows) or a range predicate, the estimator tightens its guess using the catalog (
present_count / NDV, or histogram interpolation) instead of a pessimistic whole-label count. That can move the entry point of a multi-node pattern to the genuinely cheapest node. -
Physical-rule ranking. When both a covering compound index and a selective single-property seek apply to the same node, the planner ranks them by an estimated
Cost { rows, io }rather than static priority — so a non-selective compound index correctly loses to a selective single-property seek. When the cost model overrides the default pick,EXPLAIN’s footer says so (entry.cost_override: …). -
Join ordering. A multi-pattern comma-
MATCHis reordered so the most selective pattern drives the join and the intermediate result stays small (a bounded Selinger DP for ≤ 8 patterns, greedy above that). Connectivity is preserved, so the reorder never introduces a cartesian product the written order avoided.
Because an inner join’s result is a multiset independent of join order, and a
different access path over the same data returns the same rows, cost-based
planning changes only speed, never results. A query without ORDER BY already
returns rows unordered; one with ORDER BY post-sorts.
Fast paths that make queries cheap
The executor recognizes a number of shapes and serves them without a full scan or
a blocking sort. You don’t opt into these — they fire automatically when the
shape and the available indexes match. The Physical: footer tells you which
fired.
| Shape | Fast path |
|---|---|
MATCH (n:L {p: v}) | Value-index point lookup (PropertyIndex[p] =). |
WHERE n.p IN [...] | IN-list seek — a union of point seeks, not a scan + filter. |
WHERE n.p STARTS WITH 's' | Prefix-range scan over the order-preserving string index. |
WHERE k1=X AND k2 > Y ORDER BY k2 (compound (k1,k2)) | Compound range-seek to the range boundary. |
ORDER BY p [DESC] LIMIT k (index-served) | O(log N + k) sorted scan, LIMIT pushed in. |
ORDER BY p [DESC] SKIP n LIMIT k | Deep-SKIP key-only cursor advance (no per-skipped-row fetch). |
WHERE p > $cursor ORDER BY p LIMIT k | Keyset pagination — O(log N + k) value-cursor seek. |
ORDER BY a, b LIMIT k | Covering-compound or leading-key partial-prefix scan (no post-sort). |
ORDER BY … LIMIT k (not index-served) | Bounded top-K heap — O(k) memory, not a full sort. |
traversal + ORDER BY target.prop LIMIT k | Edge-sorted top-K when a covering index exists. |
[NOT] EXISTS { (n)-[:R]->() } | Bare pattern-existence probe — one edge-index seek, no sub-plan. |
RETURN n.p, count(*) grouped by n.p | Group-by-count run-length value-index scan. |
The advisor
When a traversal + ORDER BY target.prop LIMIT k shape could be served by an
edge-sorted index but none exists, both EXPLAIN and the advisor procedure tell
you exactly which index to create:
$ patinadb ./mygraph.db query \
"EXPLAIN MATCH (c:Company {name:'Acme'})<-[:WORKS_AT]-(p:Person)
RETURN p ORDER BY p.age DESC LIMIT 3"
plan: 'Limit skip=0 count=3
Project items=1 distinct=false return_star=false
Sort keys=1
MatchScan required=[(c:Company) -[entry](p:Person) → PropertyIndex[age] SORTED SCAN DESC est.3]
Physical:
limit.bounded_topk
Advice: CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age (would serve this ORDER BY … LIMIT via edge_sorted_topk instead of a fan-out top-K)
'
The same suggestion is available programmatically:
$ patinadb ./mygraph.db query \
"CALL patinadb.advisor('MATCH (c:Company {name:\"Acme\"})<-[:WORKS_AT]-(p:Person) RETURN p ORDER BY p.age DESC LIMIT 3') YIELD suggestion"
suggestion: 'CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age'
Creating that index flips the plan to limit.edge_sorted_topk and the advice
disappears. See Edge-Sorted Indexes.
Bounded-memory guards
Blocking operators (aggregate input, non-top-K ORDER BY, UNION-distinct, and
hash-join build sides) are capped at PATINADB_MAX_AGG_ROWS (default 5,000,000).
Past the cap they raise a clear, actionable error instead of an OOM — the default
is far above any normal query, so below it the result is byte-identical. See
Configuration and Caching & Memory Tuning
for the RAM budget model, and Cache Observability & Tuning
for the read-cache hit rates that make repeated hot queries cheap.
Procedures (CALL)
patinaDB has an extensible procedure framework. Procedures are invoked with
CALL, declare a fixed set of named yield columns, and stream rows back
into the query like any other operator. A trailing CALL with no RETURN
implicitly returns all yielded columns.
CALL patinadb.engrams() YIELD id, message, timestamp
RETURN id, message ORDER BY timestamp DESC
User-defined procedures are not supported — you cannot register your own from Cypher. The framework is an internal extension point; the built-ins below are what’s available.
Built-in procedures
History & diff
| Procedure | Yields | Purpose |
|---|---|---|
patinadb.engrams() | id, message, timestamp, … | List committed engrams (newest first). |
patinadb.diff(id) | per-operation rows | Git-style view of a single engram. |
patinadb.diffRange(from, to) | structural change rows | Move-aware structural diff between two points. |
These mirror the CLI subcommands (log, diff, diff-range) and the MCP
tools (engrams, diff, diff_range). See Engrams
and Diffs.
Full-text search
Registered under two namespaces — the Neo4j-compatible name and a patinaDB alias — so existing Neo4j tooling works unchanged:
| Procedure | Alias | Yields |
|---|---|---|
db.index.fulltext.queryNodes(name, query) | patinadb.fulltext.queryNodes | node, score |
db.index.fulltext.queryRelationships(name, query) | patinadb.fulltext.queryRelationships | relationship, score |
queryNodes yields real nodes (not just property maps), so they render as
graph nodes in the Neo4j Browser and hydrate correctly through Bolt drivers:
CALL db.index.fulltext.queryNodes('docs', 'graph AND database~')
YIELD node, score
RETURN node.title, score ORDER BY score DESC LIMIT 10
See Full-Text Search for index creation and the query syntax.
Graph algorithms
A small set of read-only graph algorithms run over an in-memory snapshot of the
current graph. Each takes an optional node-label and relationship-type
projection (pass null to include everything) and yields real nodes plus a
per-node result. gds.*.stream aliases are provided for tooling discoverability
— note they use patinaDB’s simplified (label?, relType?, config?) signature,
not Neo4j GDS’s named-graph-projection API.
| Procedure | Alias | Yields | Purpose |
|---|---|---|---|
patinadb.algo.pageRank(label?, relType?, config?) | gds.pageRank.stream | node, score | Iterative PageRank. config = {iterations: 20, dampingFactor: 0.85}. Scores sum to ≈ 1.0. |
patinadb.algo.wcc(label?, relType?) | gds.wcc.stream | node, componentId | Weakly-connected components. componentId is the smallest node UUID in the component (stable across runs). |
patinadb.algo.degree(label?, relType?, config?) | gds.degree.stream | node, score | Degree centrality. config = {direction: 'both'} (in/out/both); a self-loop counts twice for both. |
patinadb.algo.betweenness(label?, relType?) | gds.betweenness.stream | node, score | Exact betweenness centrality (Brandes’ algorithm) over directed shortest paths. A path’s interior nodes score highest; O(V·E). |
patinadb.algo.closeness(label?, relType?) | gds.closeness.stream | node, score | Closeness centrality over directed shortest paths, Wasserman–Faust normalized ((k/(N-1))·(k/Σd)) so disconnected graphs are well-defined; unreachable-only nodes score 0. O(V·E). |
patinadb.algo.triangleCount(label?, relType?) | gds.triangleCount.stream | node, triangles | Per-node triangle count over the undirected simple graph (self-loops/parallel edges collapsed). Global count = Σ triangles / 3. |
patinadb.algo.labelPropagation(label?, relType?, config?) | gds.labelPropagation.stream | node, communityId | Deterministic synchronous label-propagation community detection. config = {maxIterations: 10}. communityId = the smallest node UUID in the community (stable across runs, like WCC). |
CALL patinadb.algo.pageRank('Page', 'LINKS', {iterations: 30})
YIELD node, score
RETURN node.title, score ORDER BY score DESC LIMIT 10
The algorithms materialize the projected graph in RAM (fine up to ~10M edges on one box), run in memory, and yield without mutating anything — so they need no replication and can run on any cluster node. Because PageRank is normalized to sum to 1, its ranking matches Neo4j but its magnitudes differ.
betweenness and closeness follow out-edges (directed shortest paths),
matching PageRank/degree; triangleCount and labelPropagation treat edges as
undirected. Every algorithm is deterministic — the crux for
labelPropagation, which is normally randomized: patinaDB uses a fixed
synchronous update with a lowest-label tie-break and a hard iteration cap (iterations/maxIterations are bounded at 1000 so a read call cannot become a
compute DoS), so every cluster node computes the same communities. Because
betweenness/closeness are O(V·E), they suit small-to-medium projections; the
in-memory-only scale ceiling applies. APOC utility procedures and
weighted/personalized PageRank variants are planned follow-ons.
Statistics catalog
| Procedure | Yields | Description |
|---|---|---|
patinadb.stats(label?) | label, property, count, ndv, min, max | The query planner’s statistics catalog for a label (or all labels). One summary row per label (property = null, count = vertex count) plus one row per property with its present count, distinct-value count (NDV), and min/max. |
CALL patinadb.stats('Person')
YIELD label, property, count, ndv, min, max
RETURN property, count, ndv, min, max ORDER BY ndv DESC
The catalog feeds cost-based entry-point selection: it lets the planner estimate
how selective a WHERE/pattern filter is (e.g. a rare property value vs a common
one) and drive a query from the cheapest starting point. It also feeds cost-based
join ordering: a multi-pattern MATCH (a)…, (b)…, (c)… is reordered so the most
selective pattern drives the join and intermediate results stay small, instead of
joining patterns in written order. Statistics are local performance hints —
they change only which plan runs, never the result (a different join order returns
the same rows) — so they are computed lazily per node, cached, and never
replicated.
Index advisor
| Procedure | Yields | Description |
|---|---|---|
patinadb.advisor(query) | suggestion, reason, current_plan | Analyze a query string (parsed + planned exactly as EXPLAIN) and suggest an index that would flip it to a faster physical plan. Today it surfaces missing edge-sorted indexes for a traversal + ORDER BY target.prop LIMIT k shape; empty result means nothing to advise. |
CALL patinadb.advisor(
'MATCH (c:Company {name:"Acme"})<-[:WORKS_AT]-(p:Person) RETURN p ORDER BY p.age DESC LIMIT 3')
YIELD suggestion, reason, current_plan RETURN suggestion
-- suggestion: 'CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age'
EXPLAIN prints the same advice as an Advice: footer. See
Query Planning & Performance and
Edge-Sorted Indexes.
Cache observability
| Procedure | Yields | Description |
|---|---|---|
patinadb.cache.stats() | scope, kind, bytes, entries, hit_rate, generation | The multi-level, RAM-budget-governed cache. One row per (level, scope): scope is the hot database / collection (label) / query shape (db:1/label:Ticket), kind is the cache level, and hit_rate is hits / (hits + misses). generation is reserved (yielded as null today). |
CALL patinadb.cache.stats()
YIELD scope, kind, bytes, entries, hit_rate
RETURN scope, kind, bytes, hit_rate ORDER BY bytes DESC
Read-only and node-local. When caching is disabled (the default) it yields zero
rows (no error). The same view is available over HTTP as GET /mgmt/cache and as
Prometheus gauges on /metrics. For every metric and a tuning playbook see
Cache Observability & Tuning.
CSV export
Write graph data out as neo4j-admin-style CSV — the same format the CLI
import / export commands use, so an exported file loads straight back with
patinadb import (:ID(uuid) reproduces vertex UUIDs, byte-identical).
| Procedure | Alias | Yields | Description |
|---|---|---|---|
patinadb.export.csv(label, path [, config]) | — | file, kind, rows | Export one label’s vertices to CSV. path ending .csv writes a single node file; otherwise path is a directory and gets nodes_<Label>.csv plus that label’s outgoing relationships as rels_<TYPE>.csv (one per edge type). config = {rels: false} skips relationships. |
patinadb.export.query(query, file) | apoc.export.csv.query | file, kind, rows | Export an arbitrary query result (selective export). CSV columns are the RETURN names. |
-- Whole label + its relationships into a directory:
CALL patinadb.export.csv('Person', '/data/exports/people')
YIELD file, kind, rows RETURN file, kind, rows
-- A selective slice via a query:
CALL patinadb.export.query(
'MATCH (p:Person) WHERE p.age > 30 RETURN p.name AS name, p.age AS age',
'/data/exports/over30.csv')
Node files carry an :ID(uuid) column, one typed column per property
(name:int / :float / :boolean / :date / :localdatetime, or a bare
string column) and a :LABEL column; an absent property is a blank cell.
Relationship files carry :START_ID(uuid) / :END_ID(uuid) / :TYPE + typed
property columns. Query-export entity cells (nodes/relationships) are rendered
as their UUID / type — use patinadb.export.csv for a structural round-trip.
Export streams row-by-row (bounded memory) over the current committed graph.
Security note. These procedures write a file on the machine running the query — the same file-write consideration as
LOAD CSVreading afile://URL. On the server both are deny-by-default: an export writes only to directories whitelisted with--allow-export-dir, and any file-I/O query is raised to require the global Admin role (details). Embedded and CLI use installs no sandbox and is unaffected. The procedures are CSV-only; Parquet/Arrow export goes through the CLI (patinadb export --format parquet) or the server’sGET /mgmt/export.
Availability across interfaces
Procedures work everywhere the engine runs: embedded, CLI, MCP, REST, and Bolt (including the Neo4j Browser). In the server, full-text query procedures read from the local node’s copy of the index, so reads are served without a round trip to the leader.
Full-Text Search
patinaDB supports user-defined full-text indexes with BM25 ranking, using Neo4j-compatible syntax. You create an index over chosen string properties of a label (nodes) or relationship type (edges), then query it through the full-text procedures, getting back ranked entities and relevance scores.
Creating an index
-- Over node properties
CREATE FULLTEXT INDEX docs FOR (n:Doc) ON EACH [n.title, n.body]
-- Over relationship properties
CREATE FULLTEXT INDEX mentions FOR ()-[r:MENTIONS]-() ON EACH [r.context]
-- With an analyzer
CREATE FULLTEXT INDEX articles FOR (n:Article) ON EACH [n.body]
OPTIONS { indexConfig: { `fulltext.analyzer`: "english" } }
- Only string properties are indexed (matching Neo4j). Non-string values on the listed properties are ignored.
- The index is maintained synchronously: it is updated as part of every committed write, so it is never stale on read (no eventual consistency).
Analyzers
The analyzer controls tokenization and stemming:
| Analyzer | Behaviour |
|---|---|
standard | Lowercase, tokenize on non-alphanumerics. (Default.) |
keyword | Treat the whole value as a single token (exact-match indexing). |
english | Standard + English Snowball stemming + English stop words. |
german | Standard + German Snowball stemming + German stop words. |
Managing indexes
SHOW FULLTEXT INDEXES
DROP INDEX docs
DROP INDEX <name>drops a full-text index by name. The Neo4jDROP INDEX ON :Label(prop)form (for ordinary indexes) is a different, unsupported statement and is left to the engine.
Querying
CALL db.index.fulltext.queryNodes('docs', 'graph database')
YIELD node, score
RETURN node, score ORDER BY score DESC
CALL db.index.fulltext.queryRelationships('mentions', '"exact phrase"')
YIELD relationship, score
RETURN relationship, score
The alias patinadb.fulltext.queryNodes / …queryRelationships is
equivalent. Results are ranked by BM25 (k1 = 1.2, b = 0.75).
Query syntax (Lucene subset)
| Form | Example | Meaning |
|---|---|---|
| Term | graph | Documents containing the term. |
| Implicit OR | graph database | Adjacent terms are OR’d. |
AND / OR / NOT | graph AND NOT sql | Boolean operators; AND binds tighter than OR. |
| Phrase | "graph database" | Terms in that exact order (consecutive positions). |
| Prefix | data* | Terms starting with data. |
| Fuzzy | databse~, db~2 | Edit-distance match (default / explicit distance). |
| Boost | graph^3 database | Multiply a term’s contribution to the score. |
| Field | title:graph | Restrict a term to one indexed property. |
| Grouping | (graph OR sql) AND db | Parenthesised sub-expressions. |
Prefix and fuzzy queries expand against the index dictionary and are capped at 256 expansions per term to bound cost.
In the server
CREATE FULLTEXT INDEX / DROP INDEX are replicated as Raft control
commands (like CREATE DATABASE): the definition propagates to every node and
each node builds the index from its own copy of the graph. Index definitions are
carried in Raft snapshots and rebuilt on snapshot install, so a node that joins
or restarts ends up with the same indexes. Query procedures read the local
node’s index.
This works end-to-end over Bolt, including the Neo4j Browser: create an index
and run queryNodes, and matching nodes come back as graph nodes with scores.
Limitations
- No phrase slop / proximity (
"a b"~3) — phrases must be exactly consecutive. - No highlighting / snippet extraction.
- No numeric or range queries inside the full-text string (full-text indexes cover string properties only).
- Postings are updated read-modify-write per document with no segment merging, so very high write throughput on a large indexed corpus is not the design target. See Limitations.
Vector Search
patinaDB supports vector / embedding search with Neo4j-compatible syntax, backed by an IVF-Flat (inverted-file, k-means clustering) approximate nearest-neighbour index. You store embeddings as list-of-float properties, create a vector index over them, then query it through the vector procedures, getting back ranked nodes and normalized similarity scores.
Storing vectors
A vector is an ordinary list-of-numbers property. Set it with plain Cypher:
CREATE (n:Product {id: 1, embedding: [0.12, -0.03, 0.88, 0.41]})
MATCH (n:Product {id: 1}) SET n.embedding = [0.10, -0.01, 0.90, 0.40]
When a vector index covers that (label, property), the index is maintained
synchronously on every write — inserts, updates, and deletes are reflected
immediately (no eventual consistency), on every replica.
Creating an index
CREATE VECTOR INDEX product_embeddings
FOR (n:Product) ON (n.embedding)
OPTIONS { indexConfig: {
`vector.dimensions`: 4,
`vector.similarity_function`: 'cosine'
} }
`vector.dimensions`(required) — the fixed vector length. Vectors of a different length, or non-numeric lists, are simply not indexed.`vector.similarity_function`—'cosine'(default) or'euclidean'.IF NOT EXISTSis supported.- Unknown option keys (e.g. Neo4j’s HNSW-specific params) are accepted and ignored, so existing Neo4j DDL runs unchanged.
IVF tuning (patinaDB extensions)
The index partitions vectors into nlist clusters (via k-means); a query scans
the nprobe clusters nearest the query vector. Defaults:
nlist = clamp(round(sqrt(count)), 1, 4096),
nprobe = clamp(nlist / 16, 1, 64). Override them:
OPTIONS { indexConfig: {
`vector.dimensions`: 384,
`vector.similarity_function`: 'cosine',
`patinadb.ivf.lists`: 256,
`patinadb.ivf.nprobe`: 16
} }
Higher nprobe → higher recall, more work per query. nprobe = nlist scans
every cluster (exact search).
Build the index after loading data. The k-means centroids are trained from the vectors present at
CREATE VECTOR INDEXtime. Creating an index over an empty label produces an index with no centroids, and vectors added later are then not indexed until youDROPand re-CREATEit. This is inherent to IVF (there is nothing to cluster over an empty set).
Managing indexes
SHOW VECTOR INDEXES
DROP VECTOR INDEX product_embeddings
DROP VECTOR INDEX product_embeddings IF EXISTS
SHOW VECTOR INDEXES yields name, label, property, dimensions,
similarityFunction, lists, and nprobe.
Querying
CALL db.index.vector.queryNodes('product_embeddings', 5, [0.1, -0.02, 0.9, 0.4])
YIELD node, score
RETURN node, score ORDER BY score DESC
Arguments: the index name, the number of nearest neighbours k, and the query
vector. Results are real graph nodes plus a score, ordered by descending
similarity. The alias patinadb.vector.queryNodes is equivalent. A query
vector whose length differs from the index’s dimensions is an error.
Similarity scalar functions
Two namespaced functions score a pair of vectors directly, using the same
normalized formulas as the index score:
RETURN vector.similarity.cosine([1.0, 0.0, 0.0], [1.0, 0.0, 0.0]) -- 1.0
RETURN vector.similarity.euclidean([0.0, 0.0], [3.0, 4.0]) -- 1/26
vector.similarity.cosine(a, b)→ Neo4j-normalized cosine(1 + rawCosine) / 2 ∈ [0, 1].vector.similarity.euclidean(a, b)→1 / (1 + euclideanDistanceSquared) ∈ (0, 1].
Both raise an error on a dimension mismatch or a non-numeric element, and
propagate null when either argument is null.
In the server (High Availability)
CREATE / DROP VECTOR INDEX are replicated as Raft control commands. This
is where the IVF choice matters:
patinaDB replicates a deterministic apply loop — every node applies the same operations and must converge to a byte-identical index, or the same query would return different results on different replicas. HNSW is randomized and insertion-order-dependent, so it would diverge. IVF-Flat’s k-means centroids are trained once on the leader and replicated as part of the index definition. Every node then assigns each vector to the nearest centroid by a pure, deterministic function → identical posting lists.
Index definitions (including their centroids) are carried in Raft snapshots, so a node that joins or restarts rebuilds identical posting lists from its restored graph. Ongoing writes are indexed synchronously on every node as they apply.
Works end-to-end over Bolt (including the Neo4j Browser): create an index and
run db.index.vector.queryNodes, and matching nodes come back as graph nodes
with scores.
How it works / limitations
- IVF-Flat, not HNSW. Vectors are partitioned into
nlistk-means clusters; a query scans only thenprobeclusters nearest the query, then computes the exact similarity to each candidate and keeps a bounded top-k. This is approximate: recall is tunable vianprobe(higher = better recall, more work;nprobe = nlistis exact). - Heavy one-time build. Assigning every existing vector to a centroid is
O(N · nlist · dim)over the whole label — inherent to any ANN build. Incremental maintenance per write is cheap (one nearest-centroid assignment + a few KV ops). - Brute-force candidate scan. Within the probed clusters the candidate scan
is a linear posting-list scan. It is deliberately isolated behind one internal
function (
VectorIndex::candidates), so a graph-based backend (e.g. HNSW per cluster) could replace it later without changing the query surface — but that backend would need to preserve cross-replica determinism. - Nodes only (v1). Vector indexes cover node properties; relationship vector indexes are not yet supported.
- Build after loading (see the note under Creating an index).
Spatial / Geo
patinaDB has a first-class Point type with Neo4j’s four coordinate
reference systems (CRSs), the point() / distance() /
point.distance.ellipsoidal() / point.withinBBox() functions, and a
CREATE POINT INDEX statement. Points are
stored, indexed on disk with a space-filling-curve key, and queried correctly, and
a planner fast path turns a radius/bbox query (and a kNN
ORDER BY distance(...) LIMIT k) into a curve-range seek instead of a full scan.
The Point type
A point carries a CRS (identified by a Neo4j SRID) and 2 or 3 coordinates:
| CRS | SRID | Dim | Coordinates |
|---|---|---|---|
cartesian | 7203 | 2 | [x, y] |
cartesian-3d | 9157 | 3 | [x, y, z] |
wgs-84 | 4326 | 2 | [longitude, latitude] |
wgs-84-3d | 4979 | 3 | [longitude, latitude, height] |
For geographic (wgs-84) points, x is the longitude and y is the
latitude (Neo4j’s convention), so p.latitude reads the second coordinate.
Two points are equal iff they have the same SRID and coordinates — a
cartesian and a wgs-84 point are never equal even with identical numbers.
< / > on points are undefined (they return null, matching Neo4j);
ORDER BY uses a deterministic total order (the space-filling-curve order).
point({...}) — constructing points
CRS is inferred from the map keys, or set explicitly with crs / srid:
// cartesian (x/y) — 2D and 3D
RETURN point({x: 3.0, y: 4.0})
RETURN point({x: 1.0, y: 2.0, z: 3.0})
// geographic (latitude/longitude) — 2D and 3D
RETURN point({latitude: 52.52, longitude: 13.405})
RETURN point({latitude: 52.52, longitude: 13.405, height: 100.0})
// explicit override
RETURN point({x: 13.4, y: 52.5, crs: 'wgs-84'})
RETURN point({x: 1.0, y: 2.0, srid: 4326})
Out-of-range latitude (|lat| > 90) or longitude (|lon| > 180) raises an
ArgumentError. point(null) returns null.
Points can be stored on nodes and relationships like any other property:
CREATE (:City {name: 'Berlin', loc: point({latitude: 52.52, longitude: 13.405})})
Accessors
Field access reads a coordinate or CRS component:
WITH point({x: 3.0, y: 4.0, z: 5.0}) AS p
RETURN p.x, p.y, p.z, p.crs, p.srid
MATCH (c:City)
RETURN c.loc.latitude, c.loc.longitude
.x/.longitude → axis 0, .y/.latitude → axis 1, .z/.height → axis 2,
.crs → the CRS name string, .srid → the integer SRID. Accessing a missing
component (e.g. .z on a 2-D point) returns null.
distance() / point.distance()
Both spellings work. Returns metres for geographic points (spherical Haversine) and Euclidean distance for cartesian points:
// Euclidean → 5.0
RETURN distance(point({x: 0, y: 0}), point({x: 3, y: 4}))
// Haversine, Berlin → Paris ≈ 878 km (metres)
RETURN point.distance(
point({latitude: 52.52, longitude: 13.405}),
point({latitude: 48.8566, longitude: 2.3522})
)
// radius query (served by a curve-range seek when a POINT INDEX exists)
MATCH (c:City)
WHERE distance(c.loc, point({latitude: 52.5, longitude: 13.4})) < 5000
RETURN c.name
Mixed CRS (or mismatched dimensionality) returns null, matching Neo4j.
wgs-84 distance is spherical (mean Earth radius 6 371 009 m), ~0.3 % off the
true geoid — fine for radius search, not survey work.
point.distance.ellipsoidal() — accurate WGS-84 geodesic
When you need survey-grade accuracy, use the ellipsoidal variant, which computes
the true WGS-84 geodesic distance via the Vincenty-inverse formula (accounting
for the Earth’s oblateness). distance() stays spherical Haversine (Neo4j parity);
point.distance.ellipsoidal() is the distinct, more-accurate spelling:
// WGS-84 ellipsoidal (Vincenty), Berlin → Paris ≈ 878 km — a distinct value
// from Haversine, but within 0.5 %.
RETURN point.distance.ellipsoidal(
point({latitude: 52.52, longitude: 13.405}),
point({latitude: 48.8566, longitude: 2.3522})
)
Same argument conventions as distance(): null/non-point operand → null,
mixed CRS/dimensionality → null. It is WGS-84 only — a cartesian
argument falls back to plain Euclidean (there is no ellipsoid without a geoid).
Near-antipodal pairs (where Vincenty does not converge) fall back to the
always-finite spherical distance rather than emitting NaN. It is an exact scalar
only — it does not drive the radius/bbox seek (which uses the spherical
bounding math); a radius filter should still use distance().
point.withinBBox()
Bool — whether a point lies inside an axis-aligned bounding box:
MATCH (c:City)
WHERE point.withinBBox(c.loc,
point({x: -1, y: -1}),
point({x: 10, y: 10}))
RETURN c.name
A geographic box whose lowerLeft.longitude > upperRight.longitude wraps the
antimeridian — inc 1–3 reject it with a clear error rather than silently
returning false (full antimeridian/pole handling is inc-4).
CREATE POINT INDEX
A point index is a planner-enablement marker. The on-disk curve keys are written for every point property unconditionally (see below), so registering an index is a lightweight no-op backfill; the def tells the planner it may use a curve-range seek for radius/bbox queries over that property (increment 3).
CREATE POINT INDEX city_loc [IF NOT EXISTS] FOR (n:City) ON (n.loc)
DROP POINT INDEX city_loc [IF EXISTS]
SHOW POINT INDEXES -- also folds into SHOW INDEXES
Like every other index, it replicates across a Raft cluster (re-run deterministically on each node) and is carried in snapshots.
How points are stored (the curve key)
Every point property lands in the same label-scoped value index as every other
scalar, under an order-preserving Morton / Z-order key (encode_for_index
tag 0x07):
0x07 ++ srid(u32 BE) ++ morton_interleave(order-preserving axis codes) ++ exact axis codes
Each axis’s f64 runs through the same order-preserving u64 transform the
Float index uses; the per-axis codes are bit-interleaved MSB-first into a
fixed-width big-endian string (16 bytes for 2-D, 24 for 3-D) so byte order equals
Z-curve order. The SRID leads the key, so points of different CRS occupy disjoint
ranges. The exact coordinates are appended so the point decodes back exactly. A
2-D point key is 37 bytes. This is the final on-disk format — it ships in
increment 1 so there is never a migration when the seek arrives.
Accelerating radius / bbox queries: the point-index seek
Create a point index so radius and bounding-box queries seek the Morton curve instead of scanning the whole label:
CREATE POINT INDEX loc_idx FOR (n:City) ON (n.loc)
Once the index exists, a query of the form
MATCH (c:City) WHERE distance(c.loc, point({latitude: 48.85, longitude: 2.35})) < 5000
RETURN c
or
MATCH (c:City) WHERE point.withinBBox(c.loc, point({x: 0, y: 0}), point({x: 10, y: 10}))
RETURN c
runs as a bounded set of curve-range seeks plus an exact post-filter: the
query region is decomposed into a few contiguous Morton ranges (a superset of the
answer), each is seeked in the value index, and the exact distance() /
withinBBox filter trims the false positives. Results are identical to the
full scan — the index only changes speed. EXPLAIN shows entry.spatial_seek in
the Physical: footer when the seek is used. Without a point index the query
still runs correctly, just as a label scan (Neo4j’s declare-to-accelerate model).
A radius seek is roughly an order of magnitude faster than the full scan once
the label is large and the region is selective.
Geographic edge cases are handled: a radius that crosses the ±180° antimeridian
is split into two boxes and unioned; one that reaches a pole widens to all
longitudes (a correct over-approximation the post-filter trims); an
antimeridian-wrapping withinBBox (lower-left longitude greater than
upper-right) covers [ll.lon, 180] ∪ [-180, ur.lon].
The seek also composes with a (non-kNN) ORDER BY — e.g.
MATCH (c:City) WHERE distance(c.loc, point({latitude: 48.85, longitude: 2.35})) < 5000
RETURN c ORDER BY c.name
still uses entry.spatial_seek to produce the candidate set, then applies the
ORDER BY c.name as an ordinary post-sort over just those matches (byte-identical
to a full scan + sort). Only a kNN ORDER BY distance(...) LIMIT k is served
by its own path (above).
Nearest-neighbour (kNN)
MATCH (c:City) RETURN c ORDER BY distance(c.loc, point({latitude: 48.85, longitude: 2.35})) LIMIT 10
With a point index on c.loc, this runs an expanding-ring search over the
curve: it grows a search box until the k-th nearest is provably confirmed (no
un-searched point can be closer), then orders just that candidate set — identical
to the full sort, but without touching every row. Without an index it falls back
to the exact full sort.
Polygons & geometry (geometry MVP)
Beyond points, patinaDB has a first-class Polygon value and a small set of
areal predicates — a scoped geometry MVP, not full PostGIS.
Constructing a polygon
// Exterior ring from a list of points (auto-closed if the last ≠ first):
RETURN polygon([point({x: 0, y: 0}), point({x: 10, y: 0}),
point({x: 10, y: 10}), point({x: 0, y: 10})]) AS square
// With holes — a list of rings, ring 0 = exterior, rings 1.. = holes:
RETURN polygon([
[point({x: 0, y: 0}), point({x: 10, y: 0}), point({x: 10, y: 10}), point({x: 0, y: 10})],
[point({x: 4, y: 4}), point({x: 6, y: 4}), point({x: 6, y: 6}), point({x: 4, y: 6})]
]) AS ring_with_hole
The CRS is inherited from the points (all points must share one CRS — a mixed-CRS set is an error). A ring needs ≥ 3 distinct vertices. A polygon can be stored as a node/relationship property (it round-trips through storage) but is not spatially indexed (see limitations).
Predicates
// Point-in-polygon (ray casting, correct for holes):
MATCH (p:Place) WHERE within(p.loc, $region) RETURN p
// contains() is the same predicate with arguments swapped:
RETURN polygon.contains($region, point({x: 5, y: 5})) // → true/false
// Polygon–polygon intersection (overlap or touch):
RETURN intersects($regionA, $regionB) // → true/false
within(point, polygon)/contains(polygon, point)— even-odd ray-casting point-in-polygon: inside the exterior ring and outside every hole. A point exactly on an edge/vertex is reported inside (a documented boundary convention). Also available as the namespacedpolygon.contains(polygon, point)/polygon.within(point, polygon).intersects(polyA, polyB)— a bounding-box reject fast path, then an edge-segment-crossing test, then a vertex-containment test (so containment with no crossing edges still counts). Returnstruewhen the polygons overlap or touch.- Mixed-CRS operands →
null(parity withdistance()).
Limitations (current increments)
- Geometry is a scoped MVP — a
Polygontype withwithin/contains/intersectsonly. No linestrings, no multipolygon, noST_*OGC function library, no spatial joins beyond the point predicates above. - No polygon spatial index — polygon predicates are always a full scan +
exact filter (a polygon is not added to the tag-
0x07point curve index; a polygon-column BVH / R-tree is an honest follow-on). A “points within a constant query polygon” optimization could later reuse the point bbox-seek. - Polygons are 2-D — a polygon’s footprint is
[x, y]; a 3-D point’s height is dropped on construction. - wgs-84 polygons are treated as planar lon/lat — no antimeridian-crossing and no polar geometry (a polygon spanning the ±180° seam is out of MVP scope). Ray casting / intersection assume a flat plane, which is fine for local regions but not for large geodesic areas.
intersectsassumes well-formed input — degenerate or self-intersecting polygons are undefined (not asserted).distance()is spherical Haversine, not ellipsoidal (Neo4j’s own default; ~0.3% off the true geoid — fine for radius search, not survey work). Usepoint.distance.ellipsoidal()when you need the accurate WGS-84 geodesic.- 3-D seeks are less selective than 2-D (interleaving three axes has worse curve locality), but still correct — the exact post-filter always runs.
Engrams
Every mutation in patinaDB is recorded. A engram is one committed unit of change — a list of low-level delta operations (create/delete vertex, set property, set/remove label, create/delete edge) plus metadata: an id, a parent id, a timestamp, and an optional message. The chain of engrams is the source of truth for history, diffs, time travel, and — in the server — replication.
Think of it as a git-like commit log for your graph: an append-only history you can inspect, diff, travel through, tag, and compact.
How writes become engrams
Autocommit (the common case). Every write — embedded Dataset::query, a REST
/cypher call, or a Bolt RUN — is captured and committed as one engram,
atomically. A single statement, however complex (MATCH … CREATE … SET …, or a
bulk UNWIND … CREATE), is one engram.
#![allow(unused)]
fn main() {
let ds = Dataset::open("./mygraph")?;
ds.query("CREATE (n:Person {name: 'Ada'})", None)?; // one engram
ds.query("CREATE (m:Person {name: 'Charles'})", None)?; // another engram
}
Explicit transactions. Over Bolt, BEGIN … COMMIT groups several statements
into one engram applied atomically at COMMIT (see
Bolt). ROLLBACK discards them — no engram is written.
Embedded staging. The library also exposes manual staging: begin opens an
in-memory pending engram, you stage ops, and commit applies them as one
engram.
#![allow(unused)]
fn main() {
let pending = ds.begin(Some("seed".into()));
// … stage ops into `pending` …
let meta = ds.commit(pending)?; // one engram, applied atomically
}
Listing history
CALL patinadb.engrams() YIELD id, message, timestamp
RETURN id, message, timestamp ORDER BY timestamp DESC
patinadb ./mygraph log # CLI
The MCP server exposes the same as the engrams tool.
Snapshots & compaction
To keep time travel fast, patinaDB periodically captures a full-graph snapshot (by default every 50 commits, configurable). Reconstructing a past state loads the nearest snapshot and replays deltas forward from there, rather than replaying the whole history. Snapshots are an internal optimisation — you interact with history through engrams, diffs, tags, and time travel.
Determinism & replication
A engram is a pure, deterministic description of a change: replaying a committed op stream reproduces the exact same graph, and a created vertex’s generated UUID is baked into the op so replays are stable. This is what lets the server replicate — a write becomes a Raft log entry of delta ops, and every node applies the same ops to reach the same state.
The engram lifecycle
History is a managed asset, not just an append-only ledger. These operations let you name, protect, compact, branch, and promote points in it.
Pin — protect a engram from compaction
A pinned engram is never coalesced by squash, so the point-in-time it marks stays reachable.
CALL patinadb.pin('<engram-id>')
CALL patinadb.unpin('<engram-id>')
Tag — a named, snapshotted, pinned point
A tag is a stable name for a engram (like a git tag), so you can refer to a meaningful point without tracking raw ids. Creating a tag pins its engram and takes a full snapshot there, so reading it back is cheap and squash never removes it.
CREATE TAG v1 -- tag the current HEAD
CREATE TAG release AS OF '<engram-id>' -- tag a specific engram
SHOW TAGS -- list name → engram
DROP TAG v1 -- remove (unpins if unreferenced)
Read a database as it was at a tag with
AS OF TAG. Tags
replicate across a cluster — every node names, pins, and snapshots the same
engram — so SHOW TAGS and AS OF TAG work against any node.
Squash — compact old history
Coalesce a run of old engrams into a single synthetic genesis, keeping recent history intact. Pinned (and too-recent) engrams are boundaries squash stops at. The live graph is unchanged; only the log is compacted.
CALL patinadb.squash(10) -- keep the 10 most recent, coalesce older
CALL patinadb.squash(10, 1719792000) -- …only those older than a unix timestamp
Fork — branch a database at a point
Create a new database seeded with the state of another at a chosen engram (HEAD if omitted). The fork starts with a single genesis engram and its own independent history.
FORK DATABASE prod AS OF '<engram-id>' INTO staging
Restore — promote a past state to HEAD
Bring the state at a past engram back to the live graph as a new engram. History is preserved (nothing is rewritten); the restore is an ordinary append-only write, so it replicates cleanly.
CALL patinadb.restore('<engram-id>')
This is the write-side counterpart to time travel (which is read-only): time travel reads the past, restore promotes it to the present.
Subscribing to changes (CDC)
The engram log is also a live change source: the server’s
GET /changes endpoint streams each committed engram as
it applies, resumable from an engram cursor — so external systems can react to
graph changes as they happen (cache invalidation, search-index sync, ETL).
In the cluster
All of the lifecycle operations are replicated control commands: every node
re-derives the result from its own identical history (synthetic genesis ids are
content-derived, so the ids agree on every node). SHOW TAGS / SHOW DATABASES
are local reads; the mutating commands go through the leader and need admin.
Diffs
patinaDB can show what changed — both for a single engram and between any two points in history.
Single-engram diff
A git-show-style view of one engram: what was created, deleted, and which
properties changed (old → new). Repeated SETs on the same property are
coalesced, no-op sets are dropped, and prior values are resolved by
reconstructing the parent state.
CALL patinadb.diff('<engram-id>')
patinadb ./mygraph diff <engram-id> # human-readable
patinadb ./mygraph diff <engram-id> --json
MCP: the diff tool.
Range diff (structural)
A structural diff between two reconstructed states — not a replay of the
operations between them, but a comparison of the actual graphs at from and
to. Use empty (CLI) / None as from to diff against the empty graph.
patinadb ./mygraph diff-range <from-id> <to-id>
patinadb ./mygraph diff-range empty <to-id>
CALL patinadb.diffRange('<from-id>', '<to-id>')
MCP: the diff_range tool.
Move pairing
A naive structural diff reports a node that changed identity as one removed and one added node. The range diff is move-aware: it pairs a removed and an added vertex that share a label and match on an identity property, reporting a single move instead of an add/remove pair.
The identity-property priority list defaults to
qualified_name, fqn, name. Override it on the CLI:
# Use `email` then `id` as identity; `none` disables move pairing entirely
patinadb ./mygraph diff-range <from> <to> --identity-props email,id
patinadb ./mygraph diff-range <from> <to> --identity-props none
If two candidates match ambiguously, they are left unpaired (reported as separate add/remove) rather than guessed.
Use case: code indexing
The range diff powers patinadb-indexer, which re-indexes a codebase and
reports a real per-run diff of the symbol graph (functions/types added, removed,
or moved) — even though the writes go through query() and don’t produce
per-engram deltas. That’s exactly what structural, move-aware diffing is for.
Time Travel
Because every change is recorded as a engram, patinaDB can answer read queries against the graph as it was at any past engram. The state is reconstructed (nearest snapshot + forward delta replay) into a temporary view, and your query runs against that view. The live graph is never modified.
CLI
patinadb ./mygraph query \
"MATCH (n:Person) RETURN n.name" \
--at <engram-id>
Without --at, the query runs against the current (HEAD) state.
Embedded
Dataset::query takes an optional at: Option<Uuid>:
#![allow(unused)]
fn main() {
// HEAD
ds.query("MATCH (n) RETURN n", None)?;
// As of a specific engram
ds.query("MATCH (n) RETURN n", Some(engram_id))?;
}
Server (USE … AS OF)
In the server, prefix a read with USE <db> AS OF '<engram-id>' to travel
back in a specific database:
USE sales AS OF '<engram-id>'
MATCH (o:Order) RETURN count(o)
Over REST you can equivalently pass an at field in the request body; over Bolt
the USE … AS OF prefix is parsed per query.
Semantics & constraints
- Reads only. Time travel reconstructs a read-only past view. You cannot write to the past or “restore” the database to an old state through time travel (that’s a different operation — full snapshot import/export).
- Consistent point-in-time. A time-travel query sees the entire graph as it was at that engram — a coherent snapshot, not a mix of old and new.
- Cost. Reconstruction is bounded by the distance from the nearest snapshot to the target engram. Frequent snapshots (see Engrams) keep this cheap; querying a point far from any snapshot replays more deltas.
Tags — named, snapshotted points in history
A tag is a stable name for a engram (like a git tag), so you can time-travel to a meaningful point without tracking raw engram ids. Creating a tag also pins and snapshots its engram:
- Pinned — a tagged engram is protected from squash: compaction never collapses it, so the point-in-time it names stays reachable.
- Snapshotted — a full snapshot is taken at the tagged engram, so reading
AS OFthat tag is cheap (no long delta replay), however old it is.
CREATE TAG v1 -- tag the current HEAD
CREATE TAG release AS OF '<engram-uuid>' -- tag a specific engram
SHOW TAGS -- list name → engram
DROP TAG v1 -- remove (unpins if no other tag references it)
Read a database as it was at a tag:
USE default AS OF TAG 'v1' MATCH (n) RETURN n
On a cluster, CREATE TAG / DROP TAG replicate through Raft — every node
names, pins, and snapshots the same engram — so AS OF TAG and SHOW TAGS
work against any node (including followers). SHOW TAGS is a local read;
CREATE/DROP TAG need admin and go through the leader.
Anamnesis
Anamnesis is patinaDB’s provenance/lineage system.
patinaDB is versioned by construction — the engram log already records what changed and when. Anamnesis turns that stream into a queryable W3C PROV-style property graph: for every write, who did it (an agent), the commit that did it (an activity), and — at the label / type and property level — what kind of change it made and how much.
The projection lives in a separate companion database named <db>__anamnesis,
isolated from your main graph so it never touches your graph’s trees or indexes
(main-graph read performance is unaffected). Because it is an ordinary database,
it is replicated and carried in Raft snapshots for free, and you query it with
plain Cypher.
The model
Provenance is aggregated by label/type + property, not one node per touched
vertex. A write that creates a million :Ticket rows projects a handful of
provenance nodes (one per label and property touched), not a million — the
concrete uuids are not duplicated into the PROV graph (see
Drilling to concrete uuids).
In <db>__anamnesis, each write projects to:
| Node | Stable id (uuid5 of) | Properties |
|---|---|---|
:Agent | name | name |
:Activity | engram_id | engram_id, timestamp, message? |
:NodeType | (node-label, op) | label, op |
:EdgeType | (edge-type, op) | label, op |
:Property | (on, label, key) | label, key, on |
where op ∈ {created, updated, deleted} and on ∈ {node, edge}, and the
relationships:
(:Activity)-[:WAS_ASSOCIATED_WITH]->(:Agent)(:Activity)-[:AFFECTED {count, op}]->(:NodeType | :EdgeType)(:Activity)-[:SET {count}]->(:Property)
Agent, NodeType, EdgeType and Property are upserted by a
content-derived stable id, so repeated writes never duplicate them: a
(:NodeType {label:'Ticket', op:'created'}) touchpoint is created once and
reused by every activity that creates Tickets. Each commit produces exactly one
new :Activity, with AFFECTED/SET edges from that activity carrying the
per-label counts.
Why
opis on the type node. patinaDB edges are keyed by their(outbound, type, inbound)triple with no independent edge id, so twoAFFECTEDedges from the same Activity to a shared per-label node would collide. The created / updated / deleted touchpoints are therefore distinct:NodeTypenodes (one per(label, op)), andMATCH (:NodeType {label:'X'})returns up to three of them.
How ops map to touchpoints:
AFFECTED(nodes): aCREATE→(label, created); aDELETE→(label, deleted); a property/label set or removal on a pre-existing node →(label, updated). Property/label sets on a node created in the same commit fold intocreated(creating a node with properties is one act, not a create-then-update).AFFECTED(edges): the same, keyed by the relationship type.SET(properties): every value set —SetVertexProperty/SetEdgeProperty, including on freshly created entities — records a(label, key):Propertytouch. This is the property-level tracking axis: “which activity last setTicket.status?”
The agent is the authenticated RBAC user, or "anonymous" when
authentication is disabled. The timestamp is the leader-stamped commit time
carried in the Raft entry, so it is identical on every replica.
Opt in
Provenance is off by default (zero cost when off — the write path does no provenance work). Enable it per database:
CALL patinadb.anamnesis.enable() -- turn it on for the current/USE'd db
CALL patinadb.anamnesis.disable() -- turn it off (companion data is kept)
USE sales CALL patinadb.anamnesis.enable() targets sales. Enabling creates
the sales__anamnesis companion (idempotent) and replicates to every node. Requires
the global Admin role.
Querying
The companion is a normal database — the primary path is a plain USE. Query
provenance edges in the outbound Activity → target direction:
USE mydb__anamnesis
MATCH (a:Activity)-[r:AFFECTED {op: 'created'}]->(nt:NodeType {label: 'Ticket'})
RETURN a.timestamp AS when, r.count AS how_many
ORDER BY a.timestamp DESC
“Which activities set Ticket.status, and how often?”
USE mydb__anamnesis
MATCH (a:Activity)-[s:SET]->(:Property {label: 'Ticket', key: 'status'}),
(a)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
RETURN ag.name AS who, a.timestamp AS when, s.count AS n
ORDER BY a.timestamp DESC
Query direction matters. Reading an edge property (
r.count,r.op) across an inbound<-traversal currently returnsNULL— a general engine limitation, not specific to provenance. Always traverse out of the:Activity((a)-[r:AFFECTED]->…) when you need the edge’scount/op.
A convenience read procedure returns the activities that touched a node-label (newest-first):
CALL patinadb.anamnesis('Ticket')
YIELD engram_id, timestamp, message, op, count, agent
(It is server-side sugar for the USE <db>__anamnesis MATCH … query above. The
argument is a label, not a vertex uuid — provenance is label-aggregated.)
The main-database engram log also records the writer: CALL patinadb.engrams() YIELD id, author, timestamp shows who committed each engram.
Drilling to concrete uuids
The PROV graph is deliberately coarse: it tells you which labels and properties
an activity touched and how many — not which uuids. To get the concrete
vertices a commit changed, drill into the engram delta log with the Activity’s
engram_id:
CALL patinadb.diff('<engram_id>') -- the per-vertex added/removed/changed detail
So per-vertex blame (“who last wrote this node”) is a diff-scan over history
(cost O(history)), not an O(1) lookup. A reverse uuid → activities index is a
possible future add if that access pattern becomes hot.
Enrichment
Auto-projected provenance records the structure of a write (who / what-kind /
how-many). Enrichment lets the client attach context to the write — the
run/source/model of the activity — so a pipeline can say why and from
where it wrote, not just that it wrote. This is activity-level enrichment:
each context entry becomes a property on that commit’s :Activity node.
It costs nothing unless you use it, and it is ignored (no error) when provenance is disabled for the target database.
Attaching a context
REST /cypher — add an optional provenance object (an arbitrary
key→scalar map) alongside the query:
{
"query": "CREATE (:Doc {path: 'src/main.rs'})",
"provenance": {
"agent": "indexer-run-42",
"source": "git://repo@abcd123",
"model": "text-embedding-3-large"
}
}
Bolt — drivers send it as transaction metadata, the canonical Neo4j
mechanism. patinaDB reads tx_metadata from the RUN extra (autocommit) and the
BEGIN extra (explicit transactions):
# autocommit
session.run("CREATE (:Doc {path: $p})", p="src/main.rs",
metadata={"agent": "indexer-run-42",
"source": "git://repo@abcd123",
"model": "text-embedding-3-large"})
# explicit transaction — metadata set once at begin, applies to the whole tx
tx = session.begin_transaction(metadata={"agent": "indexer-run-42",
"source": "git://repo@abcd123"})
tx.run("CREATE (:Doc {path: $p})", p="a")
tx.run("CREATE (:Doc {path: $p})", p="b")
tx.commit() # one Activity, enriched with the begin metadata
Reserved keys
Some keys are special and are consumed (never folded onto the :Activity):
agent(or its aliasactor) sets the:Agentname instead of becoming an Activity property — so the recorded agent can be a pipeline or tool (e.g."indexer-run-42","nightly-etl") rather than the authenticated RBAC user. When bothagentandactorare given,agentwins. Without either, the agent stays the authenticated user (or"anonymous").confidence,sourceandderived_fromare the provenance values. They attach at a client-chosen granularity — see Scope: where the values attach below. By default (scope: changeset) they become:Activityproperties, which is cheap.scopeselects that granularity (changeset|label|attribute|instance). It is consumed, never emitted.
Every other key becomes an ordinary :Activity property.
Querying enrichment
Enrichment props are ordinary :Activity properties, and the overriding agent is
an ordinary :Agent:
USE mydb__anamnesis
MATCH (a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
WHERE a.model = 'text-embedding-3-large'
RETURN ag.name AS agent, a.source AS source, a.timestamp AS when
ORDER BY a.timestamp DESC
Rules and limits
- Determinism. The context is folded into the projection on the Raft leader and baked into the prov ops carried in the same Raft entry, so every node records a byte-identical enriched Activity.
- Coercion (best-effort — enrichment never fails the main write). Values are
coerced to a stored scalar: strings/numbers/booleans are kept;
nullvalues are dropped; nested lists/maps are stringified to JSON. Strings are truncated to 4096 characters, and at most 32 keys are folded (excess dropped in sorted order). Malformed or oversize metadata is clamped, never rejected. - Structural keys are protected.
engram_id,timestampandmessageare owned by the projector; a context key with one of those names is ignored (it cannot overwrite the built-in Activity fields).
Scope: where the values attach
The three provenance values (source / confidence / derived_from) attach at
a granularity you choose with the reserved scope key. One write supplies
one set of values, applied uniformly to every target at that scope. This lets
you record provenance cheaply at the commit level by default, and pay the
per-entity cost only when you ask for it.
scope | Values attach to… | Cost |
|---|---|---|
changeset | the :Activity (as properties) | O(1) — default, cheap |
label | the AFFECTED edges | O(distinct labels touched) |
attribute | the SET edges | O(distinct properties) |
instance | a per-vertex GENERATED edge → :Entity | O(touched vertices) |
scope is case-insensitive; activity is an alias for changeset; an
unknown value falls back to changeset (with a warning). If none of
source / confidence / derived_from is supplied, scope is irrelevant and
nothing extra is emitted — an ordinary write pays only for the label/type
touchpoints.
Coercion is best-effort and never fails the write: confidence → float,
source / derived_from → string.
sourceis cheap by default. Because the default scope ischangeset, a baresource(orconfidence) lands on the single:Activitynode — it does not create per-entity nodes. Per-instance cost is paid only when you explicitly ask forscope: instance.
changeset (default) — commit-level
{
"query": "CREATE (:Ticket {status: 'open'})",
"provenance": { "source": "git://repo@abcd", "confidence": 0.9 }
}
source and confidence become :Activity properties (scope omitted ⇒
changeset):
USE mydb__anamnesis
MATCH (a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
WHERE a.source = 'git://repo@abcd'
RETURN ag.name AS who, a.confidence AS confidence, a.timestamp AS when
label — per touched node/edge type
{
"query": "MATCH (t:Ticket) SET t.status = 'closed'",
"provenance": { "scope": "label", "source": "bulk-migration-7" }
}
The value rides every AFFECTED edge of the write:
USE mydb__anamnesis
MATCH (a:Activity)-[r:AFFECTED]->(nt:NodeType {label: 'Ticket'})
RETURN nt.op AS op, r.count AS n, r.source AS source
attribute — per touched (label, key)
{
"query": "MATCH (t:Ticket) SET t.priority = 3",
"provenance": { "scope": "attribute", "source": "triage-rules-v2", "confidence": 0.8 }
}
The value rides every SET edge:
USE mydb__anamnesis
MATCH (a:Activity)-[s:SET]->(p:Property {label: 'Ticket', key: 'priority'})
RETURN s.count AS n, s.source AS source, s.confidence AS confidence
instance — per changed entity (the expensive opt-in)
This is the shape an extraction / LLM pipeline needs when it asserts a fact per write: “for this specific entity, which run generated it, with what confidence, from what source?”
{
"query": "CREATE (t:Ticket {summary: 'db is slow'})",
"provenance": { "scope": "instance", "agent": "run-42", "source": "doc://y", "confidence": 0.9 }
}
For each vertex created or updated in the write, in <db>__anamnesis:
(:Entity { ref_uuid, ref_label })— one per concrete main-graph entity, keyed by a uuid5 of itsref_uuid(idempotent upsert: repeated writes to the same entity reuse the node).ref_labelis the entity’s primary label.(:Activity)-[:GENERATED { confidence?, source?, derived_from? }]->(:Entity)— one per(activity, entity), carrying the reserved values as edge properties.
The reused :Agent / :Activity are the same Layer-1 nodes (instance scope
never duplicates them). Deleted vertices are skipped — per-instance
provenance of a now-deleted entity is out of scope.
Per-entity provenance — “which run generated this ticket, with what
confidence?” (read the edge props traversing out of the :Activity, or
across an inbound <- — the latter is supported):
USE mydb__anamnesis
MATCH (e:Entity {ref_uuid: '…'})<-[g:GENERATED]-(a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
RETURN ag.name AS run, g.confidence AS confidence, g.source AS source
All low-confidence entities:
USE mydb__anamnesis
MATCH (a:Activity)-[g:GENERATED]->(e:Entity)
WHERE g.confidence < 0.5
RETURN e.ref_label AS label, e.ref_uuid AS uuid, g.confidence AS confidence
ORDER BY confidence ASC
Determinism
At every scope the values are folded into the projection once on the Raft leader and baked into the prov ops carried in the same Raft entry, so the result — Activity props, edge decorations, or the per-instance graph — is byte-identical on every node.
One source per scope, per write (future: per-target)
A write supplies one set of values, applied uniformly to all targets at the
chosen scope. Per-target distinct sources — e.g. a different confidence for
each changed property under scope: attribute, or per-entity under
scope: instance, expressed as a nested map — is a possible future extension and
is not built. Likewise, field-level provenance is the finest attribute
scope offers today (per (label, key), not per (entity, key)).
Honest caveats
- Write amplification is now O(labels + properties), not O(nodes). A write projects a handful of touchpoint upserts + per-label count edges regardless of how many rows it touched. It is still non-zero work per write — keep provenance off during bulk loads if you don’t need it, and enable it afterward (the companion only reflects writes made while it was enabled).
- Per-uuid blame is a diff-scan, not a graph lookup — see Drilling to concrete uuids.
- Server path only. Provenance is projected on the Raft server write path
(REST
/cypherand Bolt). EmbeddedDataset::querywrites are a follow-up (the projector lives in the core library and is reusable viaDataset::build_provenance_ops). - Unresolvable labels bucket under
"?". A vertex property/delete op carries only a uuid; the leader resolves its label from the live graph. In the should-never-happen case that a label can’t be resolved, the touch is bucketed under label"?"(and logged) rather than dropped. - Activity ↔ engram correlation is best-effort.
Activity.engram_idis the main engram id predicted at propose time; under concurrent-write interleaving it could differ from the recorded id. The provenance graph itself is always internally consistent and byte-identical across replicas.
Command-Line Interface
patinadb operates on an embedded database directory. The first argument
is always the database path; the rest is a subcommand.
patinadb <db-path> <subcommand> [args]
If you’re running it from the Docker image rather than a host-installed
binary: docker run -v "$PWD:/data" --entrypoint patinadb patinadb/patinadb /data/<db-path> <subcommand> [args]. See Installation.
Subcommands
query
Execute a Cypher query and print the results.
patinadb ./mygraph query "MATCH (n:Person) RETURN n LIMIT 10"
patinadb ./mygraph query "MATCH (n) RETURN n" --json
patinadb ./mygraph query "MATCH (n) RETURN n" --at <engram-id>
| Flag | Effect |
|---|---|
--json | Emit results as JSON instead of a human-readable table. |
--at <id> | Time-travel: run against the graph at that engram. |
import
Offline bulk-load neo4j-admin-style CSV / Parquet / Arrow IPC files straight
into the embedded store (bypasses Raft + the engram/versioning layer — fast, but
the loaded data is HEAD-visible only, not in --at history). All node files load
first, then all relationship files.
patinadb ./mygraph import --nodes people.csv --nodes companies.parquet \
--rels works_at.arrow
patinadb ./mygraph import --nodes n.csv --rels r.csv --id-type uuid
Node files carry a :ID column, a :LABEL column, and typed property columns
(name:int, name:float, name:boolean, name:date, name:localdatetime, or
bare name for string). Relationship files carry :START_ID/:END_ID/:TYPE
- property columns. For Parquet/Arrow the property types come from the file schema. A blank/null cell means the property is absent.
| Flag | Default | Effect |
|---|---|---|
--nodes <file> | — | A node file (repeatable). |
--rels <file> | — | A relationship file (repeatable). |
--format <fmt> | (infer) | Force csv|parquet|arrow for all files (else per extension). |
--delimiter <c> | , | CSV field delimiter. |
--batch-size <n> | 10000 | Rows per durable batch. |
--skip-ref-check | off | Don’t verify rel endpoints exist (allows dangling edges). |
--id-type <t> | hash | hash = UUIDv5 of the id string; uuid = the id cell IS the UUID. |
--id-type uuid (and an auto-detected :ID(uuid) header) treat the id column as
a literal vertex UUID — this is what lets export → import reproduce the
same graph.
export
The complement of import: scan the whole graph read-only and write
import-compatible node/relationship files. One node file per primary label
(nodes_<Label>.<ext>) and one relationship file per edge type
(rels_<TYPE>.<ext>) are written into --out, streamed in --batch-size
batches (bounded memory).
patinadb ./mygraph export --out ./dump
patinadb ./mygraph export --out ./dump --format parquet
patinadb ./mygraph export --out ./dump --format arrow --batch-size 50000
| Flag | Default | Effect |
|---|---|---|
--out <dir> | — | Output directory (created if missing). |
--format <fmt> | csv | csv, parquet, or arrow. |
--batch-size <n> | 10000 | Rows per output batch (columnar formats). |
Round-trip: the id column is written as the raw vertex UUID under an
:ID(uuid) header, so re-importing the dump reproduces the same graph, the
same UUIDs included (no re-keying):
patinadb ./mygraph export --out ./dump --format parquet
patinadb ./fresh.db import --nodes ./dump/nodes_Person.parquet \
--rels ./dump/rels_KNOWS.parquet
Typed properties round-trip through the columns: Integer→:int, Float→:float,
Bool→:boolean, String, Date→:date, LocalDateTime→:localdatetime (Parquet/
Arrow use the native Int64/Float64/Boolean/Date32/Timestamp types). A
property that appears as mixed types across a label promotes (Int+Float→Float)
or, if incompatible, is written as a string column. Zoned/offset datetimes,
times, and durations are exported as ISO-8601 strings (they have no lossless
columnar type).
backup / restore
Portable dump and restore of one embedded database. backup writes a single
self-contained file (the graph plus compound-index and UNIQUE-constraint defs);
restore rebuilds a target database directory (fresh or existing) from it. This
is the embedded/offline analogue of the server’s GET /mgmt/snapshot +
POST /mgmt/restore.
patinadb ./mygraph backup ./mygraph.bak
patinadb ./mygraph backup ./mygraph.bak --history # + engram timeline (PITR)
patinadb ./fresh.db restore ./mygraph.bak
patinadb ./existing.db restore ./mygraph.bak --force # overwrite a non-empty target
| Flag | Effect |
|---|---|
--history | Also carry the full engram delta/snapshot timeline (backup only). |
--force | Required to restore into a non-empty target database (restore only). |
restore refuses to silently overwrite a non-empty target. Before opening
the backup file at all, it opens the target and cheaply checks its node/edge
counts; if the target already holds data and --force wasn’t passed, it dies
with a clear target db <path> is not empty (N nodes, M edges) — pass --force to overwrite and touches nothing (no file is opened, no data is changed). An
empty or not-yet-created target restores with no flag needed — this only
guards against accidentally pointing restore at the wrong, already-populated
directory (mirroring neo4j-admin’s --overwrite-destination convention). A
corrupt or truncated backup file also fails cleanly with a clear message
(never a panic), whether or not --force is given.
Without --history the backup is a HEAD-state dump — the current graph only.
With --history it additionally carries the engram history, so after restore
the whole pre-backup timeline is available for time-travel:
patinadb ./mygraph backup ./mygraph.bak --history
patinadb ./fresh.db restore ./mygraph.bak
patinadb ./fresh.db query "MATCH (n) RETURN count(n)" --at <old-engram-id>
changes
Print the change log — every committed engram after a cursor (oldest first),
each rendered as its individual mutations (createNode, setNodeProperty,
createRel, …). The embedded/offline analogue of the server’s /changes stream.
patinadb ./mygraph changes # whole history
patinadb ./mygraph changes --since <engram-id> # only what changed after the cursor
patinadb ./mygraph changes --json
| Flag | Effect |
|---|---|
--since <id> | Only engrams committed after this engram UUID (a cursor). |
--json | One JSON object per engram, with a changes array. |
Each engram’s own id is a valid --since cursor for the next call, so a script
can poll incrementally by remembering the last id it saw. (A built-in --follow
is intentionally not offered: an embedded database is exclusively locked by its
single process, so there is no concurrent writer for one to tail.)
algo
Run a graph algorithm via CALL patinadb.algo.* and print the top-k ranked
table (node id + value, highest first) — a convenience over typing the full CALL.
patinadb ./mygraph algo pageRank --label Person --rel KNOWS --limit 10
patinadb ./mygraph algo wcc
patinadb ./mygraph algo degree --label Person
<name> is one of pageRank, wcc, degree, betweenness, closeness,
triangleCount, labelPropagation (case-insensitive).
| Flag | Default | Effect |
|---|---|---|
--label <L> | (all) | Restrict to nodes with this label. |
--rel <R> | (all) | Restrict to relationships of this type. |
--limit <k> | 20 | Number of top-ranked rows to print. |
Spatial queries and constraints are already reachable through query, e.g.
query "CREATE CONSTRAINT FOR (p:Person) REQUIRE p.email IS UNIQUE" or
query "MATCH (p:Place) WHERE point.distance(p.loc, point({latitude:52.5, longitude:13.4})) < 5000 RETURN p".
log
List committed engrams (the history).
patinadb ./mygraph log
diff
Git-style view of a single engram.
patinadb ./mygraph diff <engram-id>
patinadb ./mygraph diff <engram-id> --json
diff-range
Structural, move-aware diff between two engrams (or from the empty graph).
patinadb ./mygraph diff-range <from-id> <to-id>
patinadb ./mygraph diff-range empty <to-id>
patinadb ./mygraph diff-range <from> <to> --identity-props email,id
patinadb ./mygraph diff-range <from> <to> --json
| Flag | Default | Effect |
|---|---|---|
--identity-props <list> | qualified_name,fqn,name | Identity priority for move pairing; none off. |
--json | — | JSON output. |
See Diffs and Time Travel for the concepts
behind diff, diff-range, and --at.
Graph Browser & Admin UI
patinadb-browser is a local web UI for exploring a patinaDB database — a graph
view, a Cypher editor, the engram history with git-style diffs and
time-travel, the Anamnesis provenance graph,
and — when pointed at a running server — an admin dashboard with cluster
health, query metrics, and database sizes. It runs as a tiny local server
(binds 127.0.0.1) and opens your browser automatically (--no-open to skip).
Ways to connect
File mode — open an embedded database directory directly (the same on-disk database the CLI and the embedded API use):
patinadb-browser ./mygraph
Server mode — connect to a running server over its HTTP
endpoint (the one that serves /cypher and /mgmt/*), authenticating with a
user set up under Authentication:
patinadb-browser --server http://localhost:8080 --user admin --password secret --database default
--database selects which database to browse (default default). In server mode
no local database is opened — every action is proxied to the node, so you can
browse a live, replicated cluster. This is the mode that unlocks the admin
dashboard below.
Bolt mode — connect to a running server over the native Bolt protocol (as any Neo4j driver would):
patinadb-browser --bolt bolt://localhost:7687 --user admin --password secret --database default
Bolt is query-only: you get the graph, table, engram-history, and Anamnesis
views (engram/diff/time-travel go through the patinadb.* procedures and
USE … AS OF), but no admin dashboard — Bolt has no management plane, so the
admin views need server mode. Node expansion in the detail panel is also
unavailable over Bolt (Bolt 4.4 doesn’t carry the stable element id). Use Bolt
mode for a quick native-protocol look; use server mode for the full tool.
The three connection flags (<db-path>, --server, --bolt) are mutually
exclusive. Common flags: --port (default 4200), --no-open.
Views
- Graph — a force-directed view of your query results; click a node to inspect its properties and expand its neighbours (incoming and outgoing relationships, with their types and properties).
- Table — the same results as rows.
- Map — a self-contained coordinate-plane view (no external map tiles) that
plots any
PointandPolygonvalues in the current result — points as dots, polygons as filled shapes, over a light graticule. Scroll to zoom, drag to pan; hover a shape for its coordinates and the owning node’s properties. Values sitting in a returned node’s property map are found automatically, soMATCH (c:City) RETURN cmaps everyc.locpoint. - Engrams — the commit history as a timeline; select an engram to see a
git-style diff (added / removed nodes and edges, and property changes as
old → new), and view the graph as of that point in history. - Anamnesis — the projected provenance graph, shown when the database has Anamnesis enabled (otherwise a short note explains how to turn it on).
Analyze & live changes
The Analyze bar above the results runs graph algorithms and paints them onto the rendered graph (all modes — file, server, Bolt):
- PageRank / Degree / Betweenness / Closeness / Triangles — a numeric score → node size plus a verdigris → amber → rust oxidation colour ramp.
- Communities (WCC) / Label Prop — a per-community colour.
Restrict any run to a node :Label and/or a :REL type. Alongside are the
Statistics catalog (label · property · count · distinct · min · max) and an
Export CSV of the current result view.
In server mode, a Live changes button opens a change-stream panel that tails the database over the node’s change-stream endpoint (Server-Sent Events): every committed write appears live as a compact list of per-change records (create/delete node & relationship, set/remove property, label changes). Pause/resume with the panel’s toggle. It is a server-mode capability and degrades to a short notice in file and Bolt mode.
Admin dashboard (server mode, admin login)
When you connect in server mode with an admin account, an Admin view appears with a live operational overview of the node and cluster:
- Cluster health — the topology: the leader, voters, and learners with their addresses, the current term and this node’s state. A single-node cluster (quorum of 1) is shown as Standalone; a multi-node cluster shows every member and reflects failover.
- Metrics — per-query-shape statistics (count, mean, p95, max), the same data
as
GET /mgmt/queries. - Storage — each database’s on-disk size (companions such as
<db>__anamnesisincluded), with the total.
The dashboard refreshes on a gentle interval. It is only available in server mode and only to administrators; a non-admin account sees the data views without it.
The browser is a convenience UI, not a security boundary — run it locally and reach a remote server over an authenticated (and ideally TLS-terminated) connection. See Authentication & TLS.
Fully offline / air-gapped
The browser makes zero external network requests — everything it needs is
bundled into the binary. The UI is a single-page Svelte app whose build inlines
all of its JavaScript, CSS, the D3 graph library, and its fonts into one
self-contained document (no CDN, no external <script>/<link>/web-font). So it
runs unchanged on a machine with no internet access or inside an air-gapped/secure
network: the graph view renders and the fonts load exactly as they do online. (The
only network traffic is to the patinaDB database or server you point it at.)
The Server (Raft)
patinadb-raft is the patinaDB server: a standalone node that holds one
or more databases, replicates writes with the Raft consensus protocol
(openraft 0.9), and exposes both a JSON REST API and the native Bolt
protocol. It scales from a single self-leading node (a plain server with no
consensus latency) up to a highly-available multi-node cluster with automatic
failover.
Running a node
patinadb-raft \
--id 1 \
--addr 127.0.0.1:21001 \
--db ./data \
--bootstrap \
--bolt-addr 127.0.0.1:7687 \
--auth-user neo4j \
--auth-password secret
| Flag | Default | Meaning |
|---|---|---|
--id <u64> | (required) | Unique Raft node id within the cluster. |
--addr <host:port> | (required) | HTTP listen address (REST + peer RPCs + management). |
--db <dir> | (required) | Database root directory — one subdirectory per database. |
--bootstrap | off | Initialize a single-voter cluster so this node leads itself immediately. |
--bolt-addr <addr> | 127.0.0.1:7687 | Bolt listener. "" disables Bolt. |
--advertised-addr | = --bolt-addr | Bolt address advertised in routing / SHOW DATABASES (set behind proxy). |
--auth-user <name> | neo4j | Username for REST Basic / Bolt LOGON / peer RPCs. |
--auth-password <p> | "" (open!) | Shared password. Empty = authentication disabled. Also PATINADB_AUTH_PASSWORD. |
--bootstrapis the easy button. One node with--bootstrapis a complete, working server — it elects itself leader and accepts writes with no further setup. You only need the management endpoints when growing to multiple nodes.
What a node exposes
- REST on
--addr— see REST API. - Bolt on
--bolt-addr— see Bolt & Neo4j Browser. - Management & peer RPC on
--addr—/mgmt/init,/mgmt/add-learner,/mgmt/change-membership,/mgmt/metrics,/health,/version, and the internal/raft/*receivers.
The replication model
Every write is turned into a deterministic batch of delta operations and
committed through Raft as one log entry. Every node applies the committed
entry to its own local graph — there is no leader/follower divergence and no
separate read-replica path for data: a read on any node serves that node’s
applied state. DDL control commands (CREATE/DROP DATABASE,
CREATE/DROP FULLTEXT INDEX) replicate the same way.
The leader resolves a Cypher write against an ephemeral mirror of HEAD to
capture the concrete ops without mutating anything locally, then proposes
Write { db, ops }; the actual mutation happens uniformly when the entry
commits and every node applies it.
Durability
Both the Raft log and the state machine are redb-backed and persistent: the
log survives restart, and last_applied is persisted so a restart resumes
without re-applying the whole log. The graph itself is already on disk. A node
can be killed and restarted and it rejoins with its state intact.
Continue with REST API, Bolt & Neo4j Browser, Multi-Database, High Availability, and Authentication & TLS.
REST API
The server exposes a small JSON-over-HTTP API on its --addr. Every route
except /health and /version requires HTTP Basic auth when a password is set
(see Authentication & TLS).
POST /cypher (alias POST /query)
Run a Cypher query. Request body:
{
"query": "MATCH (n:Person) RETURN n.name AS name LIMIT 10",
"db": "default",
"at": null,
"consistency": "local"
}
| Field | Required | Meaning |
|---|---|---|
query | yes | The Cypher (or DDL) statement. |
db | no | Target database (default "default"). Overridden by USE. |
at | no | Engram id for a time-travel read (equivalent to USE … AS OF). |
consistency | no | "local" (default) or "linearizable". See below. |
consistency controls read freshness (ignored for writes):
"local"(default) — serve this node’s applied state. Fast, causally consistent within a database, eventually consistent across replicas."linearizable"— reflect every write committed before the read began, via a leader read-index barrier. Leader-only: a follower returns503with aleader_id/leader_addrhint (like a misrouted write). Costs one intra-cluster round-trip. See High Availability.
curl -s -u neo4j:secret -X POST http://127.0.0.1:21001/cypher \
-H 'content-type: application/json' \
-d '{"query":"CREATE (n:Person {name:\"Ada\"}) RETURN n"}'
Statement routing
The handler dispatches a statement in this order:
- Full-text DDL (
CREATE/DROP FULLTEXT INDEX,SHOW FULLTEXT INDEXES) — schema commands replicate through Raft;SHOWreads the local catalog and returns{"fulltextIndexes": [...]}. - Database DDL (
CREATE/DROP DATABASE→ replicated;SHOW DATABASES→ local registry read). - A leading
USE <db>selector (optionallyUSE <db> AS OF '<id>') picks the target database / time-travel point. - Otherwise: writes go through Raft and apply on every node; reads serve from the local applied graph.
The response is JSON with the result rows (column order and per-row alignment are preserved). Errors come back as a JSON error with an appropriate HTTP status.
GET /health
Liveness check. No auth. Returns OK when the node’s HTTP server is up.
GET /version
Server version string. No auth.
Management endpoints
On --addr, for cluster operations (see High Availability):
| Endpoint | Purpose |
|---|---|
POST /mgmt/init | Initialize a cluster (alternative to --bootstrap). |
POST /mgmt/add-learner | Add a node as a non-voting learner. |
POST /mgmt/change-membership | Promote learners to voters / change the voter set. |
GET /mgmt/metrics | Raft metrics (leader, term, membership, lag). |
GET /mgmt/cache | Multi-level cache governor state: budget + per-level/-scope bytes & hit rates (see below). |
GET /mgmt/snapshot | Stream a portable, leader-anchored backup of every database (see below). |
GET /mgmt/export | Stream one database as an import-compatible CSV tar (see below). |
POST /mgmt/restore | Restore a backup into the registry (leader-only, admin; see below). |
/raft/append, /raft/vote, /raft/snapshot are the internal peer RPC
receivers — not for client use.
GET /mgmt/cache (cache observability)
Reports the node-local, RAM-budget-governed cache hierarchy. Admin-only
(it is under /mgmt/). The JSON carries the resolved budget (bytes per region:
total, cache_limit, work_mem_limit, headroom, min_free), the overall
utilization, a governor block (admission + the free-floor-vs-cap eviction-byte
split), the per-levels resident bytes / hit rates with per-scope accounting
(which database, collection (label), or query shape is hot), a sankey read-flow
block, and the governor’s last free-RAM sample (mem_available_bytes, read from
/proc/meminfo). While caching is disabled (the default) it is a truthful
all-zeros “just the OS/redb page cache” report:
{
"enabled": false,
"budget": { "total": 0, "cache_limit": 0, "work_mem_limit": 0, "headroom": 0, "min_free": 0 },
"total_resident_bytes": 0,
"utilization": 0.0,
"governor": { "admissions": 0, "rejections": 0, "evicted_free_floor_bytes": 0, "evicted_cap_bytes": 0 },
"levels": [],
"sankey": { "total_lookups": 0, "misses": 0, "layers": [ { "name": "miss", "label": "miss → storage", "value": 0 } ] },
"mem_available_bytes": 12884901888
}
An enabled node fills levels (one entry per cache layer, each with
bytes/entries/hit_rate/utilization/avg_entry_bytes/evicted_entries/
gen_invalidations/scope_invalidations + hottest-first per-scope scopes rows)
and the sankey layers with per-layer absorbed hits.
The same accounting is available in-query as CALL patinadb.cache.stats() YIELD scope, kind, bytes, entries, hit_rate, generation, and as Prometheus gauges on
/metrics (patinadb_cache_bytes{level,db}, patinadb_cache_hit_ratio{level},
patinadb_cache_evictions_total, patinadb_mem_available_bytes, …). For the full
metric reference and a tuning playbook see
Cache Observability & Tuning.
GET /mgmt/snapshot (backup / export)
Streams a portable backup of the whole registry — every database’s graph
plus its schema and access-control surface — as a single downloadable JSON
document (Content-Type: application/json, Content-Disposition: attachment; filename="patinadb-backup.json").
Query parameters:
| Param | Default | Meaning |
|---|---|---|
history | false | When true, additionally stream each database’s full engram history (delta + snapshot bodies) for point-in-time restore (PITR). See “Point-in-time restore” below. |
The backup is built and streamed in O(chunk) memory, not O(graph): each database’s graph is serialized lazily straight from its scan iterators to a temp file (the same streaming serializer the Raft snapshot build uses), and that file is streamed back as the response body and deleted once sent. The payload carries no Raft metadata (no log id / membership), so it is a portable data dump — readable and restorable into any fresh registry:
{
"databases": { "<db>": <graph snapshot> }, // graph (HEAD state)
"fulltext": { "<db>": [<index def>] }, // full-text indexes
"vector": { "<db>": [<index def>] }, // vector indexes (trained centroids)
"constraints": { "<db>": [<unique constraint>] },
"schema_constraints":{ "<db>": [<existence / node-key constraint>] },
"tags": { "<db>": [["<name>", "<engram uuid>"]] }, // tag names
"users": [<rbac user>], // global: hash + per-db roles
"engram_history": { "<db>": <full engram history> }, // ONLY with ?history=true
"heads": { "<db>": "<engram uuid>" }, // informational
"taken_at_unix_ms": <millis> // informational
}
So a /mgmt/snapshot → /mgmt/restore round-trip reconstitutes each database
including its UNIQUE / existence / node-key constraints, compound / vector
/ full-text indexes, named tags, and the RBAC user directory (argon2
hashes + per-database role grants) — not just the graph. Every schema/RBAC field
is optional, so an older backup (from before these were carried) still restores.
(restore ignores the heads / taken_at_unix_ms fields — informational labels.)
Point-in-time restore (PITR) — ?history=true
By default (?history=true omitted) the backup is a compact HEAD-state
dump: it captures each database’s current graph plus its schema and RBAC, but
not its engram delta/snapshot history. After restoring a HEAD-only backup the
graph, constraints, indexes, users, and tag names are all back, but time-travel
over pre-backup history is gone — AS OF <old-engram-id> / CALL patinadb.diff
on engrams that predate the backup are unavailable, and every restored tag
resolves to the restored-HEAD engram.
Add ?history=true to make it a PITR archive: the backup additionally
streams each database’s full engram history (engram_history above — the
per-engram delta bodies and periodic snapshot bodies). A /mgmt/restore
of such a backup reconstitutes the whole timeline, so on the restored node:
USE <db> AS OF '<old-engram-id>'reconstructs that historical state (not HEAD),USE <db> AS OF TAG <name>resolves to the tagged engram’s point in history,CALL patinadb.diff('<engram-id>')returns that engram’s historical delta,- and HEAD is the live restored graph, as always.
The history is streamed body-by-body (one delta / snapshot at a time), so a
PITR backup keeps peak memory bounded rather than materialising the whole
timeline. The engram_history field is omitted entirely from a HEAD-only backup,
and it decodes as empty on restore (#[serde(default)]), so an old backup — or a
new HEAD-only one — still restores unchanged.
Cluster caveat.
/mgmt/restoreinstalls the engram history on the node that receives the request (the leader). The HEAD graph replicates to followers via Raft, but the delta/snapshot bodies do not — so PITR is a leader-local capability. A follower that later bootstraps purely from a Raft snapshot carries only the metadata summary (the pre-existing engram-log limitation). For a single-node restore (the documented use), full PITR works everywhere.
Leader-anchored, point-in-time labelled
The export takes a leader linearizability barrier (ensure_linearizable)
before it reads anything, so:
- It is leader-only. A follower (or any non-leader) refuses the backup with a
503naming the current leader ({ "leader_id", "leader_addr", … }), the same contract as a misrouted write or a linearizable read — you can never accidentally take a stale backup from a lagging replica. - It reflects every write committed before it began, and records as of what
point each database was captured: under the barrier it captures each database’s
HEAD engram id (
heads[<db>], matchingDataset::headon the leader) plus a wall-clocktaken_at_unix_ms.
Consistency caveat (be precise). The per-database graphs are still serialized from their live state, not reconstructed as-of their pinned HEAD. A full as-of reconstruction would have to materialise each graph in memory (O(graph)), which would break the O(chunk) streaming property, so it is intentionally not done here. The practical guarantee is therefore:
- No stale-follower backups (leader barrier), and each database is labelled with the exact HEAD it was captured at.
- Cross-database point-in-time consistency holds when writes are quiesced during
the export. If writes continue while the (multi-database) backup streams, a
write that lands after the barrier can still be included in a database that is
serialized later — the recorded per-db
headstell you the intended cut, but the live bytes may run slightly ahead of it. For a guaranteed cross-database snapshot, take the backup from a quiescent cluster.
Auth-protected like every other management route (it is a full data dump): when a password is set, valid HTTP Basic credentials are required.
curl -s -u neo4j:secret http://127.0.0.1:21001/mgmt/snapshot \
-o patinadb-backup.json
GET /mgmt/export (portable CSV export)
Streams one database’s graph as a portable, import-compatible dump — the
online counterpart of the CLI export command.
GET /mgmt/export?db=<name>&format=csv
db— which database to export (defaultdefault).format—csv(the only format this release).parquet/arrowreturn400; the columnar writers live in the CLI (arrow/parquetdeps the server deliberately avoids) and a server columnar export is a documented follow-up.
The response is a tar archive (Content-Type: application/x-tar,
Content-Disposition: attachment; filename="patinadb-export-<db>.tar")
containing, in the exact neo4j-admin convention the importer reads:
nodes_<Label>.csv— a:ID(uuid)column, one typed column per property (age:int,score:float, …), and a trailing:LABELcolumn.rels_<TYPE>.csv—:START_ID(uuid),:END_ID(uuid),:TYPE, and one typed column per edge property._schema.json— a sidecar carrying the database’s UNIQUE / existence / node-key constraint defs plus compound / full-text / vector index defs, so an exported database can be fully reconstituted. (/mgmt/snapshotnow also carries these — see above — so both paths preserve the schema;/mgmt/exportis single-database + CSV,/mgmt/snapshotis whole-registry + JSON and additionally carries RBAC users and tags.)
Because the id column is the raw vertex UUID under an :ID(uuid) header, a
re-import reproduces the same graph with byte-identical UUIDs (the importer
uses the cell literally instead of hashing it). Round-trip:
curl -s -u neo4j:secret \
"http://127.0.0.1:21001/mgmt/export?db=default&format=csv" -o export.tar
mkdir out && tar xf export.tar -C out
patinadb ./restored import out/nodes_*.csv out/rels_*.csv
Like /mgmt/snapshot, it takes a leader linearizability barrier (so it is
leader-only and reflects every committed write; a follower answers 503 with a
leader hint) and streams in O(chunk) memory (per-label CSVs are written
row-by-row to a temp dir, then tar’d to a temp file that is streamed back and
deleted). Auth-protected — it is a /mgmt/ route, so it needs admin
credentials.
POST /mgmt/restore (restore / import)
Restores a backup produced by GET /mgmt/snapshot into the registry. Send it
to the leader (every step is a replicated write, so a follower answers with a
503 leader hint) and it needs admin credentials (it’s a /mgmt/ route).
curl -s -u neo4j:secret -X POST http://127.0.0.1:21001/mgmt/restore \
-H 'content-type: application/json' \
--data-binary @patinadb-backup.json
# → {"databases": 2, "ops_applied": 12345, "users_restored": 3}
For each database in the payload it issues CREATE DATABASE (idempotent), then
streams the graph back as chunked replicated writes (so the restore commits
through Raft and appears on every node), then re-registers, in order: the
full-text index defs, the compound-index defs, the vector-index defs
(with their trained centroids), the UNIQUE + existence/node-key constraints
(rendered back to CREATE CONSTRAINT … IF NOT EXISTS DDL and replayed via the
same replicated path interactive constraint DDL uses), and the tag names
(re-created pointing at the restored HEAD). Finally it restores the RBAC user
directory — each user (argon2 hash replayed verbatim, so the original password
still works) plus its per-database role grants. Every step is a replicated,
idempotent write, so a multi-node cluster converges and re-running the same
backup is a no-op — ideal for loading a backup into a fresh (empty) cluster.
It does not wipe pre-existing data first, so restore into an empty registry
(or a database that doesn’t yet hold conflicting ids) for a clean result.
What restore does NOT bring back. The backup is a HEAD-state dump, not a PITR archive (see
/mgmt/snapshotabove): engram history is not carried, so after a restore, time-travel over pre-backup engrams and pre-backup diffs are gone, and each restored tag resolves to the single restored-HEAD engram (not its original point in history). The anamnesis-enabled flag is not re-toggled on restore (the<db>__anamnesiscompanion database itself rides the backup as ordinary data, but automatic projection must be re-enabled withCALL patinadb.anamnesis.enable()). The backup is leader-anchored and HEAD-labelled, but each database’s graph is serialized from live state rather than reconstructed as-of its pinned HEAD, so cross-database point-in-time consistency only holds when writes are quiesced during the export.
Bolt & Neo4j Browser
The server speaks the native Bolt protocol, so the official Neo4j drivers and the Neo4j Browser connect to patinaDB directly. This has been validated end-to-end against Neo4j driver 6.2.0 (over both raw TCP and WebSocket) and the Neo4j Browser (node creation plus path/relationship graph visualisation).
Connecting
The Bolt listener (default 127.0.0.1:7687) auto-detects the transport: a raw
Bolt TCP handshake or a WebSocket upgrade (the Browser’s JavaScript driver speaks
Bolt inside WebSocket binary frames). Both work on the same port.
Neo4j Browser: open browser.neo4j.io, connect to bolt://localhost:7687,
authenticate with --auth-user / --auth-password.
Official driver (Python):
from neo4j import GraphDatabase
drv = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "secret"))
with drv.session() as s:
s.run("CREATE (a:Person {name:$n})", n="Ada")
for rec in s.run("MATCH (n:Person) RETURN n"):
print(rec["n"])
drv.close()
Nodes, relationships, and paths come back as proper Bolt graph types
(Node / Relationship / Path), so drivers hydrate them as graph objects and
the Browser visualises them.
Spatial values over Bolt
A patinaDB Point is sent as a native Neo4j Point struct, so official
drivers decode it as a first-class point (neo4j.spatial.Point / a driver’s
Point type):
- a 2-D point → PackStream struct tag
0x58with fields{srid: Integer, x: Float, y: Float}; - a 3-D point → tag
0x59with an addedz: Float.
The srid is the CRS discriminant patinaDB already stores — cartesian 7203,
cartesian-3d 9157, wgs-84 4326, wgs-84-3d 4979 (the same SRIDs Neo4j
uses). For wgs-84 the point’s x is the longitude and y the latitude.
A point can also be passed as a parameter: send $center as a 0x58/0x59
struct and it decodes back into a point you can use in Cypher, e.g.
RETURN distance($center, n.loc).
Polygons have no native Bolt type. Bolt only defines Point2D/Point3D, so a
patinaDB Polygon is returned as a Map —
{type: "Polygon", srid: Integer, rings: [[[x, y], …], …]} (rings[0] is the
exterior ring, rings[1..] the holes). A driver therefore sees a plain map, not
a spatial object; decode it yourself if you need the geometry.
Authentication
Auth is checked at LOGON (scheme: "basic"). An unauthenticated connection
cannot run queries. See Authentication & TLS.
Streaming reads
Read results stream lazily: the query runs on a worker thread that pushes
hydrated records over a bounded channel, with PULL n batching and backpressure.
Memory stays bounded regardless of result size, so you can stream large result
sets without materialising them all server-side.
Transactions
Both autocommit and explicit transactions are real.
- Autocommit (a bare
RUN): each statement commits on its own as one engram — atomic, even for a multi-clause or bulk statement. - Explicit (
BEGIN … COMMIT/ROLLBACK): statements betweenBEGINandCOMMITare buffered, not committed one by one.BEGINpins a consistent snapshot of the database; everyRUNsees that snapshot plus the transaction’s own uncommitted writes (repeatable reads + read-your-own-writes);COMMITapplies the whole buffer as one atomic engram (a single Raft entry on a cluster) after a conflict check;ROLLBACKdiscards it — nothing is written.
Explicit transactions run at snapshot isolation (SI). A transaction is bound
to one database — a mid-transaction USE <other> is rejected — and schema/admin
statements (e.g. CREATE INDEX, CREATE DATABASE) run eagerly, not buffered. The
official Neo4j drivers’ managed transaction functions work as expected, including
automatic retry on the conflict error below.
Isolation: explicit transactions are snapshot-isolated, not serializable
BEGINpins the database HEAD (the serialization point). Every read in the transaction observes the graph as of that moment — a commit by another connection made after yourBEGINis invisible to your transaction (repeatable reads, no phantoms). Your own buffered writes are visible to your own reads (read-your-own-writes).COMMITis first-committer-wins. Before proposing the buffer, the server checks whether any write committed since your pinned snapshot touched an entity/property your transaction also wrote. If so, theCOMMITis rejected with a transient error (Neo.TransientError.Transaction.LockClientStopped), which the Neo4j drivers’ managed-transaction functions retry automatically. This prevents lost updates (e.g. two clients that both “read a counter, increment, write it back” — one commits, the other retries against the new value instead of silently clobbering it).- Autocommit (a bare single-statement
RUN) holds the per-database write lock for the statement, so autocommit writes to one database are serialized.What SI does not give you: write skew
Snapshot isolation detects write-write conflicts only. A read-write conflict with disjoint write-sets (write skew) is still possible: two transactions can each read a value the other overwrites, as long as they write different things, and both commit. This is the standard, well-understood SI limitation — it is not serializable and not linearizable. If you need a constraint that spans rows one transaction reads and another writes (e.g. “at most one on-call engineer”), enforce it in a single autocommit statement or with an application-level guard. See Limitations.
Conflict granularity
Property writes conflict at
(entity, property)granularity (two transactions updating different properties of the same node do not falsely conflict); create/delete and label changes conflict at whole-entity granularity. Freshly-created nodes carry new UUIDs, so concurrent inserts never conflict.
Database selection
The target database is taken per-RUN / per-BEGIN (from the Bolt db field or
a USE <db> prefix) and is never sticky across autocommit statements —
connections are pooled, so each statement resolves its own database. See
Multi-Database.
System / introspection shim
On connect, the Neo4j Browser fires admin/introspection statements that are not
ordinary Cypher (CALL dbms.components(), SHOW DATABASES, CALL db.labels(),
routing-table lookups, …). The server recognises these and returns
canned/registry-derived results so the Browser connects cleanly, shows a server
version, and populates its database dropdown. Real procedures (notably
CALL db.index.fulltext.*) are not swallowed by the shim — they reach the
engine and return real data.
Cluster-aware routing
neo4j:// (routing) drivers ask the server for a routing table and then connect
to the addresses it returns. In a cluster, patinaDB answers with the real
topology — writes to the leader, reads spread across the followers — so a
routing driver offloads reads to replicas automatically. See
High Availability → Cluster-aware Bolt routing for the
full WRITE/READ/ROUTE breakdown and the lag-immune clean-tag read pattern. A
single-node server returns itself for all three roles.
Behind a reverse proxy
Behind a TLS-terminating proxy or load balancer, set --advertised-addr to the
public host:port so the routing table sends drivers to a reachable endpoint
(each node advertises its own). For a direct bolt:// connection this isn’t
needed — the default advertises the listen address. See
Authentication & TLS.
Change Streams (CDC)
patinaDB can stream every committed graph change as it happens, so external systems can react to writes — invalidate a cache, sync a search index, feed a downstream ETL pipeline. This is Change Data Capture (CDC), and it is built directly on the engram log: every commit is an engram of resolved delta-ops, and the change stream is simply that log made subscribe-able.
CDC is a server (Raft) feature, exposed over HTTP as Server-Sent Events (SSE).
The endpoint
GET /changes?db=<name>&since=<engram-uuid>
db— the database to observe (defaultdefault).since— a resume cursor: the engram id of the last change you already processed. The stream first replays every engram committed after that id, then live-tails new commits. Omit it (or pass0) to start from the beginning of history.
The response is an SSE stream (Content-Type: text/event-stream). Each committed
write arrives as one change event:
id: 6f9c…-a1b2 ← the engram id = your next resume cursor
event: change
data: {"engram_id":"6f9c…-a1b2","parent_id":"…","db":"default",
"timestamp":1751900000,"author":"alice",
"changes":[{"op":"createNode","id":"…","label":"Person"},
{"op":"setNodeProperty","id":"…","key":"age","value":30}]}
The SSE id: field is the engram id, so a standards-compliant SSE client resumes
automatically via Last-Event-ID after a dropped connection; you can also pass
it back explicitly as ?since=.
Change records
Each event’s changes array holds one compact record per delta-op:
op | fields |
|---|---|
createNode / deleteNode | id, label (on create) |
createRel / deleteRel | start, end, type |
setNodeProperty / setRelProperty | id or start/end/type, key, value |
removeNodeProperty / removeRelProperty | id or start/end/type, key |
setNodeLabels | id, labels (the full secondary-label set) |
Resume & delivery semantics
Delivery is at-least-once with the engram cursor. On reconnect with
since=<cursor>, the stream replays exactly the engrams after that cursor and
then continues live. The server subscribes to the live feed before reading
history, so no commit can slip through the gap between “read the past” and “start
tailing”; the small replay/live overlap is de-duplicated internally, so a
well-behaved consumer sees every engram after its cursor, with no gap and only
a bounded, self-healing overlap.
Persist the last engram_id you successfully processed. If your consumer
restarts, reconnect with it as since= and you continue exactly where you left
off.
Lag
The stream is backed by a bounded in-memory buffer per database. A consumer that
falls too far behind receives a lagged event:
event: lagged
data: {"resumeFrom":"6f9c…-a1b2"}
When this happens the server automatically re-reads the engram log from your last
cursor (so no changes are lost) and continues. The lagged event is only a
cue that a resync occurred.
Consistency (read this)
The change stream is node-local and eventually consistent, exactly like a
consistency=local read:
- It observes this node’s applied HEAD as it advances. On a follower it lags the leader slightly; it never reflects an uncommitted write.
- Ordering within a database is the engram chain order (each event’s
parent_idis the previous event’sengram_id). - Publishing a change never blocks replication — a slow or dead subscriber can never stall the write path; it just lags and resyncs.
For a single-writer-of-record consumer, subscribe to the leader (see cluster routing); a follower stream is fine for best-effort reactions where a small delay is acceptable.
Authorization
/changes is authorized as a read on the target database — the caller needs
at least the Reader role for that db (see
Authentication & RBAC). An unauthenticated request is rejected
with 401.
Example
# Tail the default database's changes (with credentials + resume cursor).
curl -N -u alice:apw \
'http://localhost:8080/changes?db=default&since=6f9c…-a1b2'
-N disables curl’s buffering so events print as they arrive. Any SSE-capable
client (browser EventSource, an SSE library in your language) works the same
way.
Related: CDC is a live view over the same engram log that powers diffs and time travel. Where those read history on demand, CDC pushes each new engram as it commits.
Multi-Database
A single server node hosts multiple named databases, each an isolated graph
with its own history. The --db flag points at a root directory; each
database lives in its own subdirectory underneath it. A database named
default always exists; system is reserved.
DDL
CREATE DATABASE sales
CREATE DATABASE IF NOT EXISTS sales
DROP DATABASE sales
DROP DATABASE IF EXISTS sales
SHOW DATABASES
- Database names are case-insensitive, may be backtick-quoted, and are kept filesystem-safe.
CREATE DATABASE/DROP DATABASEare replicated as Raft control commands, so the set of databases is consistent across the cluster.DROP DATABASEis idempotent and refuses to dropdefault.SHOW DATABASESis a local registry read (also surfaced over Bolt for the Neo4j Browser’s database dropdown).
Selecting a database
Per query, choose the target database in any of three ways:
USE sales
MATCH (o:Order) RETURN count(o)
- A leading
USE <db>prefix (alsoUSE <db> AS OF '<id>'for time travel). - The
dbfield in a REST request body or the BoltRUN/BEGINmetadata. - Otherwise the
defaultdatabase.
Selection is per-statement and never sticky across pooled Bolt connections.
Isolation
Databases are fully isolated — a write to default is invisible to sales and
vice versa; there is no cross-database query or traversal. Each database has its
own engram history, its own full-text indexes, and its own snapshots.
This isolation holds across failover: in a multi-node cluster, CREATE DATABASE
replicates to all nodes, writes route to the correct isolated graph, and both
databases survive a leader kill with post-failover writes still routing
correctly (this is covered by the failover test suite).
High Availability
A single --bootstrap node is already a working server, but a quorum of one has
no redundancy. Add nodes to get automatic failover: if the leader dies, the
survivors elect a new one and writes continue.
Cluster sizing
Raft tolerates failures up to a quorum. Use an odd number of voters:
| Voters | Tolerates failures | Notes |
|---|---|---|
| 1 | 0 | --bootstrap; a plain server. |
| 3 | 1 | The usual minimum for real HA. |
| 5 | 2 | Higher availability, more replication. |
Growing a cluster
Start one node with --bootstrap, then add peers as learners (they catch up
without voting) and promote them to voters:
# Node 1 (bootstrap leader)
patinadb-raft --id 1 --addr 127.0.0.1:21001 --db ./n1 --bootstrap \
--bolt-addr 127.0.0.1:7687 --auth-password secret
# Nodes 2 and 3 (no bootstrap — they join)
patinadb-raft --id 2 --addr 127.0.0.1:21002 --db ./n2 \
--bolt-addr 127.0.0.1:7688 --auth-password secret
patinadb-raft --id 3 --addr 127.0.0.1:21003 --db ./n3 \
--bolt-addr 127.0.0.1:7689 --auth-password secret
Then, against the leader’s --addr:
# Register the new nodes as learners (id → its HTTP addr)
curl -u neo4j:secret -X POST 127.0.0.1:21001/mgmt/add-learner \
-H 'content-type: application/json' -d '{"id":2,"addr":"127.0.0.1:21002"}'
curl -u neo4j:secret -X POST 127.0.0.1:21001/mgmt/add-learner \
-H 'content-type: application/json' -d '{"id":3,"addr":"127.0.0.1:21003"}'
# Promote to a 3-voter membership
curl -u neo4j:secret -X POST 127.0.0.1:21001/mgmt/change-membership \
-H 'content-type: application/json' -d '[1,2,3]'
All nodes must share the same
--auth-password— peer RPCs carry the same Basic credentials.
Auto-join with --join
Instead of running /mgmt/add-learner by hand, start a fresh node with
--join <member> (the HTTP address of any current member). It registers itself
as a learner (read replica) on startup, following a leader hint if --join
points at a follower:
# Node 1 leads; nodes 2 and 3 auto-join as learners.
patinadb-raft --id 1 --addr 127.0.0.1:21001 --db ./n1 --bootstrap ...
patinadb-raft --id 2 --addr 127.0.0.1:21002 --db ./n2 --join 127.0.0.1:21001 ...
patinadb-raft --id 3 --addr 127.0.0.1:21003 --db ./n3 --join 127.0.0.1:21001 ...
--join is mutually exclusive with --bootstrap. The nodes join as learners;
promote them to voters when you want them to count toward quorum:
curl -u neo4j:secret -X POST 127.0.0.1:21001/mgmt/change-membership \
-H 'content-type: application/json' -d '[1,2,3]'
Failover behaviour
- Election timeout is 750–1500 ms with a 250 ms heartbeat. When the leader stops heartbeating, a survivor wins an election and takes over.
- Clients connected to the dead node reconnect to a survivor (
neo4j://routing drivers do this automatically — see Bolt on--advertised-addr). - Committed writes are durable on a quorum and survive the failover; in-flight uncommitted writes to the dead leader may need to be retried.
The failover test suite exercises exactly this: a 3-node cluster, a write through the leader, kill the leader, and assert the survivors elect a new leader, writes continue, and the cluster retains all records — including the multi-database case. A companion test drives sequential writes plus linearizable reads across a single leader kill and checks that no acknowledged write is lost across the failover and that the linearizable-read count is monotonic (never decreases). This is a solid failover + acked-write-durability smoke test.
Scope of the durability test. The linearizable-read test issues writes sequentially, uses idempotent MERGE (which masks any accidental double-apply), and kills the leader once — there are no concurrent clients, no network partitions, and no operation reordering. It verifies that acked writes survive a single failover and that read counts stay monotonic; it is not a linearizability check in the Jepsen sense. True concurrent-history linearizability testing (Knossos/Elle-style history checking under concurrency, partitions, and reordering) is future work.
Removing a dead voter
A permanently-unreachable voter stays in the quorum set and blocks further membership changes. Evict it on the leader with a live-quorum guard:
curl -u neo4j:secret -X POST 127.0.0.1:21001/mgmt/evict-voter \
-H 'content-type: application/json' -d '{"id": 3}'
The endpoint demotes-then-removes the voter in one membership change and refuses
(409) if doing so would leave the surviving voters unable to form a quorum. See
Day-2 Operations for the full
contract, plus the /health vs /ready probes and request-id tracing.
Learners as read replicas
A learner replicates the log and applies it locally but does not vote. Since any node serves reads from its own applied state, learners act as asynchronous read replicas. The single-node, learner, and voter cases are all the same binary and the same apply path.
Cluster-aware Bolt routing (reads → followers, writes → leader)
A Neo4j routing driver (neo4j:// scheme) asks the cluster for a routing
table and then load-balances: it sends writes to a WRITE server and spreads
reads over the READ servers. patinaDB answers that request — over both the native
Bolt ROUTE message and the dbms.routing.getRoutingTable procedure — with the
real cluster topology:
| Role | Servers returned |
|---|---|
| WRITE | the leader’s advertised Bolt address (only it writes) |
| READ | every non-leader member (followers + learners) |
| ROUTE | all members (any node can answer a routing request) |
The effect: neo4j:// clients automatically offload reads to the followers
and send writes to the leader — read scaling with no application changes. On a
single-node --bootstrap cluster the leader is the only member, so all three
roles resolve to that one node (no change from a standalone server).
Each node learns its peers’ advertised Bolt addresses from a background poll
of every peer’s GET /version (which now reports advertised_bolt_addr) — the
same poller that tracks peer protocol versions. Because the addresses returned to
drivers are the --advertised-addr values, set that to each node’s public
host:port when running behind a TLS-terminating proxy, so routing stays
reachable (see Bolt). If the leader is momentarily unknown (an
election is in flight) WRITE falls back to the local node — the client then gets
the leader-hint 503 on the misrouted write and retries.
A follower read is eventually consistent (it serves that node’s local applied state, which may lag the leader). That is exactly what you want for analytics and browsing. When you need a read that is both lag-immune and free to load-balance across followers, use the clean-tag pattern below.
Read consistency
By default a read serves the local applied state of whichever node you hit:
- Causally consistent within a database — the Raft log is a total order, so a single node never sees writes out of order.
- Eventually consistent across replicas — a follower or learner that lags the leader’s commit index may not yet reflect a write that has already committed elsewhere. Reads are fast (no cluster round-trip).
When you need a read to reflect every write committed before it began, opt in
to a linearizable read on the REST /cypher endpoint:
curl -u neo4j:secret -X POST 127.0.0.1:21001/cypher \
-H 'content-type: application/json' \
-d '{"query":"MATCH (n:Person) RETURN n","consistency":"linearizable"}'
This routes through a leader read-index barrier: the leader confirms it is
still leader (a heartbeat to a quorum), waits until it has applied the current
commit index, then runs the read. Only the leader can serve a linearizable
read — sending one to a follower returns a 503 whose body names the leader
(leader_id / leader_addr), exactly like a misrouted write.
It costs one intra-cluster round-trip; the default "local" read skips it.
consistencyis a REST-only knob today. The Bolt path uses the default local read; driver-level causal consistency (bookmarks) is not yet implemented.
Lag-immune reads across followers: the clean-tag pattern
A linearizable read pins you to the leader, which defeats read scaling. When you want a consistent, lag-immune read that can still load-balance freely across followers, read as of a named tag instead:
USE mydb AS OF TAG 'nightly-2026-07-04'
MATCH (n:Person) RETURN n
An engram tag names a specific point in history. Tags are
replicated, snapshotted, and deterministic, so reading AS OF TAG '<name>'
returns bit-identical results on every node regardless of replication lag — a
follower that is behind on new writes still reconstructs the tagged state
exactly. This gives you a stable, reproducible read that any follower can serve:
- Point routing-driver reads at the followers (automatic — see above).
- Tag a known-good state (e.g. after a nightly load) and have reporting/analytics
read
AS OF TAGthat tag. - Every replica agrees on the answer, and no read has to touch the leader.
See Time Travel and Engrams for creating and
managing tags. Use consistency: linearizable (above) only when you specifically
need read-your-latest-write against the live HEAD.
Rolling upgrades
Nodes in a cluster exchange a versioned wire/disk protocol: the Raft log entry
payload (AppRequest), its response, and the streamed-snapshot record format.
That format is append-only and versioned, which is what makes a rolling
upgrade (upgrade one node at a time, no full-cluster downtime) safe as long as
you upgrade in the right order.
What the format guarantees
- Append-only log-entry variants. New kinds of replicated command are only
ever appended to the
AppRequestenum, never inserted or reordered. Log entries are stored positionally (bincode), so reordering would silently re-map already-persisted entries; a build-time test pins the exact order and count to prevent it. A newer node can therefore always decode an older node’s entries. - Observable protocol version. Each node reports a
protocol_versionon itsGET /versionendpoint. It bumps whenever a new variant (or other wire/disk change) lands, so you can confirm what every node speaks before and during an upgrade. - Automatic capability gate. The leader will not propose a command that
some cluster member is too old to apply. Each node polls every peer’s
GET /versionin the background and tracks the cluster-wide minimum protocol version. Before a command is appended to the Raft log, the leader checks the version that command requires against that minimum: if any member is older — or its version hasn’t been confirmed yet (conservative: unknown is treated as too old) — the proposal is rejected without being written (409on REST, a failure with the same message on Bolt) telling you to finish the upgrade first. Today every command is protocol v1, so the gate always passes; it is future-proofing that makes the next new command safe automatically, in any node-upgrade order. - Loud rejection of an unknown variant. If an older node receives a peer
RPC carrying a command it does not know how to decode (because a newer leader
emitted it), it logs a clear error — “unknown AppRequest variant — this node
is older than the leader; upgrade it” — and returns
422, instead of a silent or opaque failure. A mixed-version wedge is diagnosable from the logs. - Snapshot format guard. The streamed snapshot carries a
format_versionfor its record stream. On install, a node refuses an unknown (newer) version before clearing its graph, so a version-skewed snapshot can never tear a half-restored store — the node keeps its existing data and can retry once upgraded.
Upgrade order
Upgrade the nodes one at a time (each catches back up before you move on); the
leader can be upgraded last or stepped down first. You no longer have to time it
perfectly: the capability gate blocks any command a not-yet-upgraded member
couldn’t apply, so exercising a new feature too early fails cleanly (409 /
Bolt failure, “upgrade all nodes first”) instead of wedging a node. Once every
node reports the new protocol_version on GET /version, the gate opens on its
own and the new functionality just works.
Check GET /version on every node to see when they all agree on
protocol_version.
Scope of the guarantees
The capability gate is the leader-side safeguard: it refuses to emit a command
until it has confirmed every member can apply it (unconfirmed peers block
conservatively). The append-only variant discipline and the loud 422 on an
unknown variant remain the defence in depth if a command ever does reach an older
node, and a version-skewed snapshot is still refused before it can clear a graph.
Finer-grained, per-feature negotiation (beyond a single monotonic protocol
version) is possible future work.
Day-2 Operations
Once a cluster is up, three things make it operable: a probe that tells a load balancer or Kubernetes when a node can actually serve, request correlation so a slow query is traceable end-to-end, and a way to remove a permanently-dead voter so it stops blocking membership changes and quorum math.
Health vs. readiness probes
The server exposes two unauthenticated probe endpoints. Both are auth-exempt
(like /version / /metrics) so a load balancer or orchestrator can reach them
without credentials, and neither is ever shed by the concurrency limiter.
| Endpoint | Semantics | Point it at |
|---|---|---|
GET /health | Liveness — 200 whenever the process is up, always. | Kubernetes liveness probe (restart the container if it stops answering). |
GET /ready | Readiness — 200 only when the node can actually serve. | Kubernetes readiness probe / load-balancer health check / neo4j:// read rotation. |
/ready returns 200 with {"ready": true, …} only when all of the
following hold, and otherwise 503 with a short machine-readable reason
(checked in priority order):
reason | Meaning |
|---|---|
no_leader | No known leader (fresh node, or mid-election). |
installing | The node is installing a Raft snapshot (graph being rebuilt). |
degraded | Community-mode write-degrade: the node couldn’t phone home within the grace window (see Licensing & Telemetry). |
lagging | last_log_index − last_applied exceeds --readiness-max-lag (default 50) — this replica is behind. |
shedding | The concurrency limiter (--max-concurrent-requests) is saturated. |
Example bodies:
// GET /ready → 200
{ "ready": true, "current_leader": 1, "last_applied": 42, "apply_lag": 0 }
// GET /ready → 503 (fresh node, no leader yet)
{ "ready": false, "reason": "no_leader", "current_leader": null, "last_applied": 0, "apply_lag": 0 }
Because /ready fails when a node is leaderless, lagging, mid-install, or
degraded, pointing a load balancer at it keeps stale/failing reads out of the
rotation. Keep /health as the liveness probe so a slow-but-catching-up node is
not restarted while it recovers.
The distroless image ships without a shell, so the compose
healthcheckcan’tcurlfrom inside the container — probe/healthand/readyfrom the orchestrator / an external monitor instead.
Kubernetes example
livenessProbe:
httpGet: { path: /health, port: 8080 }
periodSeconds: 10
readinessProbe:
httpGet: { path: /ready, port: 8080 }
periodSeconds: 5
Request correlation & tracing
Every HTTP request runs inside a tracing span carrying a request id:
- If the caller sends an
X-Request-Idheader it is honored (used verbatim); otherwise a fresh UUID is minted. - The id is echoed on the response
X-Request-Idheader, and it appears on every log line emitted while handling the request — so a slow or failing query is traceable from the client through the server logs. - An inbound W3C
traceparentheader’strace_idis picked up into the span astrace_id, for OpenTelemetry-compatible correlation across services.
Setting --otel-endpoint <url> (or PATINADB_OTEL_ENDPOINT) forces
structured JSON logging on (regardless of PATINADB_LOG) so an OpenTelemetry
Collector’s filelog receiver — or any log shipper — can ingest each event with
its span context (request_id, trace_id). This is log-based correlation;
a native OTLP span exporter is a documented follow-on. With no endpoint set,
the X-Request-Id correlation still works — it’s always on.
Telemetry-degrade observability and break-glass
A community-mode node that can’t reach the telemetry endpoint for the whole grace window (default 72h) degrades: client writes are refused until a heartbeat succeeds again (see Licensing & Telemetry). Two things make that freeze operable instead of a surprise:
-
Alert before it happens.
/metricsexposespatinadb_telemetry_degraded(0/1, always present) andpatinadb_telemetry_seconds_until_degrade(present only while community mode is armed and not yet degraded) — wire a Prometheus alert on the latter dropping below, say, one hour so you learn about a telemetry-endpoint outage well before writes actually stop. -
A time-boxed break-glass override, for the rare case where you need writes to keep flowing through a telemetry outage you can’t fix immediately:
patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap \ --auth-password "$PW" \ --telemetry-degrade-override-until 72hFlag
--telemetry-degrade-override-until <value>/ envPATINADB_TELEMETRY_DEGRADE_OVERRIDE. The value is always resolved to an absolute deadline, so the override must expire — accepted forms:Form Example Resolves to Duration from startup 72h,3d,30m,90snow + durationRFC3339 UTC timestamp 2026-07-12T09:00:00Zthat instant Bare unix-second integer 1799999999that instant While the override is active,
/metricsalso exposespatinadb_telemetry_override_until_seconds(seconds remaining before the override itself expires) so you don’t lose track of it. The override is logged loudly at startup — it is a deliberate, visible escape hatch, not a silent bypass. Once it expires, the normal grace-window degrade behavior resumes exactly as if it had never been set.
Tenant isolation and read-proc guards
Two related operational knobs, covered in full elsewhere but worth knowing about when running a shared/multi-tenant cluster:
--rbac-closed(database-level deny) stops a non-admin’s global role from reaching every database by default — see Authentication & TLS: database-level deny.- Expensive read procedures are bounded, not unlimited.
CALL patinadb.algo.betweenness/closeness(O(V·E), Reader-callable) are guarded by a cooperative deadline under--query-timeout-secs(the algorithm checks in periodically and bails cleanly instead of running past the timeout to completion) plus a static work budget,PATINADB_MAX_ALGO_WORK(env, default ~1e10 — see Configuration Reference), so a Reader can’t pin a blocking thread indefinitely even with no--query-timeout-secsset. Not yet implemented: a bounded blocking-thread pool and a per-user concurrent-read-procedure cap — today’s guards stop a single expensive call from running forever, but a burst of many concurrent expensive calls from different users is not yet rate-limited.
Removing a dead voter
A permanently-unreachable voter stays in the quorum set until you remove it, and it can block further membership changes. In a 3-voter cluster, one dead voter plus one more failure is a quorum loss — so evict a node you don’t intend to bring back.
curl -u admin:… -X POST http://<leader>/mgmt/evict-voter \
-H 'content-type: application/json' -d '{"id": 3}'
# → 200 {"ok": true, "evicted": 3, "voters": [1, 2]}
- Admin-only (under
/mgmt/), and must be issued on the leader — a follower returns a503leader hint (like a misrouted write). - It performs a demote-then-remove in one joint-consensus membership change.
- Quorum guard: the request is refused (
409) if the resulting voter set’s reachable members (probed over/health, self always counted live) could no longer form a majority — the endpoint won’t hand you a cluster that can’t commit. Evicting an unknown id, or the last remaining voter, is a400.
Auto-eviction after a failure-detector timeout, and learner→voter
auto-promotion, are follow-ons — this manual/admin endpoint is the current MVP.
Use GET /mgmt/cluster to see the live voter/learner topology before and after.
Authentication & TLS
Authentication
The server uses a single shared credential (--auth-user / --auth-password,
or the PATINADB_AUTH_PASSWORD environment variable).
-
Enabled when a password is set. An empty password means no authentication, which is fail-closed: the node refuses to start unless you also pass
--insecure-disable-auth. This prevents accidentally exposing an open node by forgetting to set a password. With the flag, the node runs open and logs a prominent warning — only acceptable on a trusted, firewalled, single-tenant network (ideally still behind a TLS-terminating proxy).# Refuses to start (no password, no opt-in): patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap # → Error: refusing to start with authentication disabled: set --auth-password … # Deliberately open (trusted network only): patinadb-raft --id 1 --addr 127.0.0.1:21001 --db ./data --bootstrap \ --insecure-disable-auth -
REST: HTTP Basic on every route except the open probes
/healthand/version(and/metricsonly if you opt out of metrics auth — see below). -
/metrics: authenticated by default — the exposition series carrydb=<name>labels, so an open endpoint would let an unauthenticated scraper enumerate every database name and its per-db volume. It sits behind the same Basic auth as every other route. To serve it open on a private, trusted monitoring network, pass--insecure-open-metrics(or setinsecure_open_metrics: true/ the legacyrequire_metrics_auth: falsein the config file)./healthand/versionstay open regardless. -
Bolt: checked at LOGON (
scheme: "basic"); an unauthenticated connection cannot run queries. -
Peer RPCs: the
/raft/*inter-node calls authenticate with a dedicated cluster secret (--cluster-secret/PATINADB_CLUSTER_SECRET, sent in theX-Cluster-Secretheader), separate from the root-admin credential so the two rotate independently. When it’s empty the node falls back to the admin password, so single-credential deployments are unchanged — but all cluster nodes must share the same cluster secret (or password, in the fallback case).
patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap \
--auth-user neo4j --auth-password "$PATINADB_AUTH_PASSWORD"
Users & roles (RBAC)
Beyond the shared credential, the server supports per-user accounts with roles.
The configured --auth-user / --auth-password is the built-in root admin
(always accepted — you can’t lock yourself out); additional users are created at
runtime and replicate across the cluster.
Roles are global and ordered by privilege:
| Role | Can do |
|---|---|
reader | read-only Cypher (MATCH … RETURN) |
writer | reads and writes (CREATE/SET/DELETE/MERGE) |
admin | everything: writes, all DDL, cluster /mgmt/*, user management |
Manage users via Cypher-style DDL (admin only), e.g. over REST:
# As the root admin:
curl -u neo4j:secret -X POST http://127.0.0.1:21001/cypher \
-H 'content-type: application/json' \
-d "{\"query\":\"CREATE USER alice SET PASSWORD 'apw' SET ROLE writer\"}"
# alice can now write but not manage users or the cluster:
curl -u alice:apw -X POST http://127.0.0.1:21001/cypher -d '{"query":"CREATE (n:Person)"}' ... # 200
curl -u alice:apw -X POST http://127.0.0.1:21001/cypher -d '{"query":"CREATE USER eve …"}' ... # 403
CREATE USER <name> SET PASSWORD '<pw>' [SET ROLE <role>](default rolereader)ALTER USER <name> SET PASSWORD '<pw>' | SET ROLE <role>DROP USER <name>,SHOW USERS
User changes replicate through Raft (passwords are argon2-hashed on the leader)
and are carried in snapshots. Enforcement applies to both REST and Bolt. Wrong
credentials → 401; insufficient role → 403.
Per-database roles
A user has a global default role plus optional per-database overrides. The effective role on database X is the override for X if set, otherwise the global role — so an override can both elevate and restrict a user on a specific database:
# alice is a global reader…
curl -u neo4j:secret … -d "{\"query\":\"CREATE USER alice SET PASSWORD 'apw' SET ROLE reader\"}"
# …but a writer on the `sales` database only:
curl -u neo4j:secret … -d "{\"query\":\"GRANT writer ON DATABASE sales TO alice\"}"
# revoke it again:
curl -u neo4j:secret … -d "{\"query\":\"REVOKE ON DATABASE sales FROM alice\"}"
GRANT <role> ON DATABASE <db> TO <user>REVOKE [<role>] ON DATABASE <db> FROM <user>
Per-database roles govern data reads/writes on that database. Cluster
management (/mgmt/*), database/user DDL, and GRANT/REVOKE themselves always
require the global admin role. SHOW USERS reports each user’s db_roles.
Database-level deny (closed-mode tenant isolation)
By default the role lattice bottoms out at reader — every authenticated,
non-admin user can USE any database and read it, including a database it was
never granted anything on. For a genuinely multi-tenant deployment, opt into
closed-mode RBAC:
patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap \
--auth-password "$PW" --rbac-closed
- Flag
--rbac-closed/ envPATINADB_RBAC_CLOSED/ config keyrbac_closed. Off by default — enabling it is a real behavior change: a non-admin whose only credential is a global role loses access to every database it holds no explicit grant on (a per-database role override, or a per-label grant). That loss of blanket access is the isolation. - A global
admin, the root credential, and (if you disabled auth entirely) the open-node case are always unaffected — closed mode only narrows non-admin access. - Enforced identically on both transports — REST (
server::authorize_data, which also covers the/changesCDC stream) and Bolt (against the resolvedUSE-target) — with aNeo.ClientError.Security.Forbidden(403/ Bolt failure) on a denied database. - Anamnesis companions are bound to their base database. A
<db>__anamnesisprovenance companion has no grants of its own; closed mode resolves it back to<db>for the authorization check — a user who can readsalescan readsales__anamnesis, and a user with no access tosalescannot reach its companion either. A label-scoped user is authorized against the base db’s label grants, which it typically doesn’t hold for the companion’s synthetic PROV labels — so companions fail closed for label-scoped users unless explicitly granted.
Combine it with per-database roles or per-label grants (below) to give each tenant exactly the databases/labels it needs, with everything else invisible.
Per-label grants (fine-grained RBAC)
For finer control than a per-database role, grant a user READ or WRITE on a
specific label in a database:
# alice may read Person nodes in `sales`, and write Ticket nodes there:
curl -u neo4j:secret … -d "{\"query\":\"GRANT READ ON sales:Person TO alice\"}"
curl -u neo4j:secret … -d "{\"query\":\"GRANT WRITE ON sales:Ticket TO alice\"}"
# revoke one privilege:
curl -u neo4j:secret … -d "{\"query\":\"REVOKE WRITE ON sales:Ticket FROM alice\"}"
GRANT READ|WRITE ON <db>:<Label> TO <user>REVOKE READ|WRITE ON <db>:<Label> FROM <user>
A WRITE grant implies READ (you can’t write a label you can’t read).
When a user has any per-label grant for a database, data queries against that
database are authorized per label instead of by the blanket db-role. The
query’s touched node labels are extracted and checked: every label it reads
needs READ, every label it writes (CREATE / MERGE / SET n:Label /
REMOVE n:Label / DELETE) needs WRITE. A user with only a (global or per-db)
role and no label grants is unaffected — full db access exactly as before.
Enforcement is REJECT, not row-filtering: a query that touches an ungranted
label — or an unclassifiable node set (an all-graph MATCH (n), a CALL
procedure that reads arbitrary labels, an unlabelled CREATE, or a DELETE
whose target label isn’t statically known) — is refused with
Neo.ClientError.Security.Forbidden (403 on REST, a failure over Bolt). The
label extractor is default-deny: anything it cannot statically classify is
rejected, so a missed label can never become a silent grant. Enforcement is
identical on REST and Bolt (a shared code path) and on every node (grants
replicate through Raft and are snapshot-carried, so each replica decides the same
way). SHOW USERS reports each user’s label_grants (db → { label → "r"/"rw" }).
REJECT vs row-filtering. patinaDB rejects a whole query that touches an ungranted label; it does not transparently filter out the ungranted rows and run the rest (Neo4j’s property/label “traverse/read” filtering model). Row- level filtering, and per-property privileges, are documented follow-ons.
Security audit log
Every authenticated write / admin / DDL operation and every authorization
denial is recorded to a node-local audit log (who, when, action, database,
allow/deny, and a literal-collapsed statement fingerprint — so passwords in
CREATE USER never appear). Both transports feed it: the REST /cypher
choke-point and the Bolt RUN authorization choke-point both record through
the same shared dispatch::classify + audit machinery, so a denial or a
write/DDL issued over a raw Bolt connection (a driver, or the Neo4j Browser)
shows up in the trail exactly like a REST one — Bolt is no longer a blind spot.
Read it back, newest-first (admin-only):
curl -u neo4j:secret http://localhost:8080/mgmt/audit?limit=100
Each event is also emitted to tracing (target patinadb::audit) for a durable,
centralized trail via your log pipeline. Scope (honest): the /mgmt/audit ring is
in-memory + bounded + node-local (not Raft-replicated, not persisted across a
restart). Successful reads are not recorded (writes + denials only). Persisted /
replicated audit and read-operation auditing are follow-ons.
Encryption-at-rest for the audit trail and the graph is a storage-backend
concern and is out of scope — use OS-level disk encryption today.
TLS
Native TLS for the HTTP plane
Pass --tls-cert + --tls-key (PEM) to serve the REST / management / Cypher
API and the inter-node Raft RPCs (they share --addr) over HTTPS. Peers then
talk https:// to each other and verify the certificate.
patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap \
--tls-cert /etc/patinadb/server.pem \
--tls-key /etc/patinadb/server.key \
--tls-ca /etc/patinadb/ca.pem \
--auth-password "$PATINADB_AUTH_PASSWORD"
- The certificate’s SANs must cover the peer
--addrhosts (IPs/hostnames a peer dials), or peer verification fails. --tls-cais the CA peers verify each other with — for a self-signed / private-CA cluster, the cert (or CA) that signed every node’s--tls-cert. Omit it when node certs chain to a public CA (system roots are used).- TLS is opt-in: with no flags, the node serves plaintext (use the reverse proxy stance below).
The same --tls-cert/--tls-key also secures the Bolt endpoint: it
terminates TLS before dispatching, so native drivers (bolt+s:// / neo4j+s://)
and the Neo4j Browser (wss://) both connect over TLS. The certificate must cover
the Bolt host clients dial (see --advertised-addr for routing behind a proxy).
Reverse-proxy termination (alternative / for Bolt)
You can instead terminate TLS at a reverse proxy (nginx, Caddy, HAProxy, a cloud load balancer) — required for encrypted Bolt today:
- Terminate
https://in front of the REST--addr(or use native TLS above). - Terminate
bolt+s:///neo4j+s://(or wss for the Browser) in front of the Bolt--bolt-addr.
--advertised-addr behind a proxy
neo4j:// routing drivers fetch a routing table and then connect to the address
the server advertises. Behind a proxy, the listen address is not the address
clients should use, so set --advertised-addr to the public host:port:
patinadb-raft --id 1 --addr 0.0.0.0:21001 --db ./data --bootstrap \
--bolt-addr 127.0.0.1:7687 \
--advertised-addr graph.example.com:7687 \
--auth-password "$PATINADB_AUTH_PASSWORD"
Now routing sends drivers to graph.example.com:7687 (your proxy), which
terminates TLS and forwards to the node. For a direct bolt:// connection with
no proxy, leave --advertised-addr unset (it defaults to --bolt-addr).
Hardening against malformed input
The Bolt wire decoder (packstream) parses attacker-controlled bytes before
authentication succeeds — a HELLO/LOGON handshake is unauthenticated by
definition. A length-prefixed PackStream List/Map/String used to pre-allocate a
buffer sized directly from the untrusted length field, so a handful of crafted
bytes claiming a huge length could drive a large-allocation denial-of-service
before a single credential was checked. The decoders now bound every
pre-allocation by the remaining input size, so a claimed length can never
allocate more than the bytes actually available — a malformed/truncated frame
errors cleanly instead of pinning memory. This class of decoder is also under
continuous fuzzing (cargo +nightly fuzz run packstream_unpack /
bolt_request, see the repository’s patinadb-raft/fuzz) to catch regressions
before they ship.
Trust domains
A node exposes several surfaces with different trust expectations. Treat them as distinct and firewall accordingly:
| Surface | Port | Auth | Trust domain |
|---|---|---|---|
Client REST (/cypher, /mgmt/*) | --addr | Basic → per-user RBAC | application / operators |
| Bolt | --bolt-addr | LOGON → per-user RBAC | drivers / Neo4j Browser |
Peer RPC (/raft/*) | --addr | cluster secret | other cluster nodes only |
/metrics | --addr | Basic by default (open with --insecure-open-metrics) | monitoring stack |
/health, /version | --addr | open | load balancers / probes |
The peer-RPC surface shares the port with the client REST surface but is a
cluster-internal trust domain — it should only be reachable from the other
nodes, never the public internet. The cluster secret is the boundary; rotate it
independently of the admin password. Enable HTTP-plane TLS (--tls-cert /
--tls-key / --tls-ca) so peer RPCs and client traffic are encrypted and
peers authenticate each other’s certificates.
RBAC changes propagate through Raft (eventual on followers)
User and grant changes (CREATE/ALTER/DROP USER, GRANT/REVOKE) are
replicated log commands, not local edits: the leader hashes the password
(argon2) and proposes the record, and every node applies it on commit. Two
consequences:
- A change is durable and cluster-wide once committed, but a follower only reflects it after it applies that log entry — a just-created user may be briefly unknown on a replica that hasn’t caught up. Authenticate writes and admin against the leader (or use a linearizable read) if you need read-your-own-grant immediately.
- The root admin (
--auth-user/--auth-password) is not replicated — it’s local config accepted directly on every node, so you can always authenticate even before the user directory has replicated (and you can’t lock yourself out of a node by dropping users).
Cypher-driven file I/O (LOAD CSV / export procs)
Two Cypher features touch the server’s local filesystem: LOAD CSV FROM 'file://…' reads a host file, and the CSV export procedures
(patinadb.export.csv / patinadb.export.query / apoc.export.csv.query) write
one. On the server these are locked down by two independent layers — both
must pass:
-
Role: global Admin. File I/O is authorized by effect, not by whether the query mutates the graph. A
LOAD CSV(a read) and the export procs (declaredProcMode::Read) both require the globaladminrole — a per-database Reader or Writer is refused (403on REST, a Bolt failure). Graph-only procedures (algorithms, statistics) are unaffected and stay Reader-level. -
Path sandbox (deny-by-default). The requested path must canonicalize to a location strictly under a configured allow-directory:
--allow-csv-dir <dir>— permitted directories forLOAD CSVreads (mirrors Neo4j’sdbms.directories.import).--allow-export-dir <dir>— permitted directories for export writes.
Both are repeatable and default-deny: with none configured the server refuses all Cypher file I/O. Paths are canonicalized before the check, so
..traversal and symlinks that escape the allow-directory are rejected. TOCTOU-hardened open (Linux): the canonicalize check and the actual file open are two separate syscalls, which in principle leaves a symlink-swap window between them. On Linux, the sandbox closes it by re-verifying the already-open file descriptor’s real path (via/proc/self/fd/N) is still under an allow-directory before any bytes are read or written — a race that swaps a symlink after the initial check is caught and the operation aborts before data crosses the boundary. (On other platforms the check falls back to canonicalize-then-open; a fullopenat2(RESOLVE_BENEATH)is a documented follow-up.)
Embedded library / CLI are unaffected. The sandbox is only installed by the server process; the embedded
patinadblibrary and thepatinadb-cliimport/exportcommands run with the caller’s own privileges and no confinement — there is no remote attacker in that trust boundary.
Example: allow reads from /srv/import and writes to /srv/export only:
patinadb-raft --id 1 --addr 127.0.0.1:21001 --db /var/lib/patinadb --bootstrap \
--auth-password "$PW" \
--allow-csv-dir /srv/import \
--allow-export-dir /srv/export
Then, authenticated as an admin:
LOAD CSV WITH HEADERS FROM 'file:///srv/import/people.csv' AS row
CREATE (:Person {name: row.name});
CALL patinadb.export.query('MATCH (n:Person) RETURN n.name AS name',
'/srv/export/people.csv');
A path outside those directories (e.g. file:///etc/passwd, or
/srv/export/../../etc/cron.d/x) is denied by the sandbox, and a non-admin is
denied by the role gate before the query runs — no file is touched either way.
Deployment checklist
- Set a strong
--auth-password(via env var, not a flag in shell history). - Set a distinct
--cluster-secret(env var) shared by every node. - Bind
--addr/--bolt-addrto localhost or a private interface; expose only through the TLS proxy. - Restrict the peer-RPC /
/metricssurfaces to the cluster + monitoring network./metricsis authenticated by default; only pass--insecure-open-metricswhen it’s confined to a trusted monitoring network. - Enable TLS (
--tls-cert/--tls-key/--tls-ca) for any multi-node cluster — otherwise the cluster secret + client credentials cross the wire in cleartext (the node warns about this at startup). - Same
--auth-passwordand--cluster-secreton every cluster node. -
--advertised-addr= the public endpoint when using routing behind a proxy. - Back up the
--dbdirectory (it holds the graph, the Raft log, and snapshots). - Leave
--allow-csv-dir/--allow-export-dirunset unless you needLOAD CSV/ export procs — file I/O is deny-by-default and requires the globaladminrole. When you do set them, point at dedicated, isolated directories (never a path holding secrets, config, or the--dbdir).
Language Bindings
Beyond the CLI and server, patinaDB is reachable from Python, an MCP server for AI agents, and a graph browser / admin UI.
Python (patinadb)
A native Python extension (PyO3). The API is shaped for a data-science workflow: open a database, bulk-load a DataFrame, query it, run a graph algorithm, and get results back as ordinary Python objects.
Install the wheel provided with your distribution (or from your organization’s package index):
python -m venv .venv && . .venv/bin/activate
pip install patinadb-<version>-<platform>.whl
Open, query, close
The class is Database, opened via Database.open(path):
from patinadb import Database
db = Database.open("./mygraph") # opens or creates an embedded db
db.query("CREATE (n:Person {name: 'Ada', age: 36})")
# query_rows → list[dict], keyed by RETURN column (the pandas-friendly path)
rows = db.query_rows("MATCH (n:Person) RETURN n.name AS name, n.age AS age")
# rows == [{"name": "Ada", "age": 36}]
db.close()
query() returns patinaDB’s column-oriented QueryResult shape;
query_rows() returns row-oriented list[dict] with native Python values
(a node/relationship cell becomes a nested dict). Both accept named parameters:
db.query_with_params(
"MATCH (n:Person) WHERE n.name = $name RETURN n", {"name": "Ada"}
)
db.query_rows("MATCH (n:Person) WHERE n.age > $min RETURN n.name", {"min": 30})
Bulk-load a DataFrame → graph
bulk_load_nodes / bulk_load_edges take a list of dicts — exactly what
DataFrame.to_dict("records") produces — and stream them through the engine’s
batched, durable bulk-insert path. Loaded data is visible at HEAD (it bypasses
the versioning layer, so it is not in AS OF time-travel reads).
import pandas as pd
people = pd.read_parquet("people.parquet") # columns: user_id, name, age
follows = pd.read_parquet("follows.parquet") # columns: src, dst
# Nodes: deterministic UUIDs from a business key (id_key) → idempotent re-loads.
db.bulk_load_nodes("Person", people.to_dict("records"), id_key="user_id")
# Edges: 'from'/'to' are node UUIDs. Database.node_id() reproduces the id.
edges = [
{"from": Database.node_id("Person", str(r.src)),
"to": Database.node_id("Person", str(r.dst))}
for r in follows.itertuples()
]
db.bulk_load_edges("FOLLOWS", edges)
Graph algorithms
Thin wrappers over the built-in patinadb.algo.* procedures, returning the
same list[dict] shape:
ranked = db.page_rank("Person", "FOLLOWS") # -> list of {node, score}
components = db.wcc("Person", "FOLLOWS") # -> list of {node, componentId}
degrees = db.degree("Person", "FOLLOWS", "in") # -> list of {node, score}
You can also call any procedure (full-text, vector, stats, export, …) directly
through query / query_rows via CALL.
API surface
| Method | Purpose |
|---|---|
Database.open(path) | Open or create an embedded database. |
db.query(cypher) | Run a query → column-oriented QueryResult. |
db.query_with_params(cypher, params) | Same, with a dict of named parameters. |
db.query_rows(cypher, params=None) | Row-oriented list[dict] (DataFrame-friendly). |
db.bulk_load_nodes(label, rows, id_key=None) | Bulk-load node records; deterministic UUIDs from id_key. |
db.bulk_load_edges(rel_type, edges) | Bulk-load edge records (from/to UUIDs). |
Database.node_id(label, key) | Reproduce the deterministic UUID for a business key. |
db.page_rank(label=None, rel_type=None, iterations=20, damping_factor=0.85) | PageRank → list[{node, score}]. |
db.wcc(label=None, rel_type=None) | Weakly-connected components → list[{node, componentId}]. |
db.degree(label=None, rel_type=None, direction="both") | Degree centrality → list[{node, score}]. |
db.close() | Close and flush. |
The bindings talk to an embedded database directly. Time-travel/engram methods (
AS OF, diffs) are reachable from the CLI, MCP, and the server; exposing them on theDatabasepyclass is planned.
MCP server (patinadb-mcp)
A Model Context Protocol server (stdio JSON-RPC) that exposes a database to an AI agent as tools. Beyond the core graph operations it surfaces patinaDB’s differentiators — time-travel, diffs, provenance, statistics, and graph algorithms:
| Tool | Purpose |
|---|---|
schema | Inspect the graph schema (labels, types, properties). |
cypher | Run a read query. |
write | Run a write query. |
begin | Start a pending engram (transaction). |
commit | Commit the pending engram. |
rollback | Discard the pending engram. |
snapshot | Capture/inspect a full-graph snapshot. |
engrams | List the versioned commit history. |
diff | Single-engram git-style diff. |
diff_range | Structural diff between two engrams. |
time_travel_query | Read the graph AS OF a past engram. |
provenance | Read the Anamnesis PROV projection (<db>__anamnesis). |
statistics | Data-shape catalog (CALL patinadb.stats) before querying. |
graph_algorithm | Run pageRank / wcc / degree / betweenness / closeness / triangleCount / labelPropagation. |
spatial_search | Nearest nodes to a point within a radius (geo metres or cartesian). |
constraints | List (SHOW CONSTRAINTS) or create a unique / exists / node_key constraint. |
advise | EXPLAIN plan + index suggestions (CALL patinadb.advisor), read-only. |
cache_stats | Per-level cache hit-rate / bytes (CALL patinadb.cache.stats). |
import_csv | Batched, deterministic-UUID CSV bulk-load (HEAD-only). |
This lets an agent query, mutate, inspect history, time-travel, and analyze a
patinaDB graph through structured tool calls. Full-text and vector search are
reachable through the cypher tool via CALL.
spatial_search takes a label, a point-typed property, and a query point as
either lat+lon (WGS-84, distance in metres via Haversine) or x+y
(cartesian, Euclidean), plus an optional radius and limit; it returns the
nearest nodes first with their distance. A CREATE POINT INDEX on the property
turns it into a fast curve-range seek. Point/Polygon cells render as compact
JSON so an agent can parse the coordinates. Label/property names are validated as
plain identifiers before inlining, so the generated Cypher can’t be injected.
Security note. The MCP server has no read-only mode and no import sandbox today — every tool listed above, including
write(arbitrary CREATE/SET/DELETE/DETACH DELETE) andimport_csv(bulk-load from any filesystem path the process can read), is available to whatever agent holds the stdio connection. If you point an LLM agent atpatinadb-mcp, treat it like handing that agent a database admin credential — there is no--read-onlyflag or import-directory allowlist to fall back on yet. Run it only against a database (and from an environment) you’re comfortable letting an agent mutate freely, and keep prompt-injected query results in mind: a malicious result returned bycyphercould steer a subsequentwritecall.
Graph browser & admin UI (patinadb-browser)
patinadb-browser is not a WASM module — it is a single axum HTTP binary
that serves one self-contained index.html (inline CSS/JS) and connects to a
database in one of three modes:
| Mode | Flag | Data plane |
|---|---|---|
| File | <db-path> | opens an embedded Dataset directly |
| Server | --server http://host:8080 | proxies a patinadb-raft node’s REST API |
| Bolt | --bolt bolt://host:7687 | native Bolt client (neo4rs) |
# File mode (embedded — no server needed)
patinadb-browser /path/to/mydb
# Server mode (against a running patinadb-raft node)
patinadb-browser --server http://127.0.0.1:8080 --user admin --password … --database default
# Bolt mode
patinadb-browser --bolt bolt://127.0.0.1:7687 --user neo4j --password …
Then open http://127.0.0.1:4200. It renders a force-directed graph view with a
Neo4j-style node detail panel and neighbor expansion, an engram/diff timeline,
graph-algorithm visualization, and (in server mode) a cluster/metrics/cache
admin dashboard. See Graph Browser & Admin UI for details.
Configuration Reference
A consolidated reference for the knobs across the CLI and the server.
CLI (patinadb-cli)
patinadb <db-path> <subcommand> [flags]
| Subcommand | Flags |
|---|---|
query | --json, --at <engram-id> |
log | — |
diff | <engram-id>, --json |
diff-range | <from> <to>, --identity-props <list>, --json |
<from> accepts empty for the empty graph; --identity-props defaults to
qualified_name,fqn,name (none disables move pairing).
Server (patinadb-raft)
| Flag | Default | Notes |
|---|---|---|
--config <path> | off | Load settings from a YAML config file (see below). |
--print-config-schema | off | Print a JSON Schema for the config file to stdout and exit. |
--id <u64> | required | Unique Raft node id (here or in the config file). |
--addr <host:port> | required | HTTP (REST + management + peer RPC). |
--db <dir> | required | Database root (one subdir per database). |
--bootstrap | off | Self-init a single-voter cluster. |
--join <member> | off | Auto-join an existing member as a learner on startup. Excludes --bootstrap. |
--bolt-addr <addr> | 127.0.0.1:7687 | Bolt listener; "" disables Bolt. |
--advertised-addr <a> | = --bolt-addr | Public Bolt address for routing behind a proxy. |
--auth-user <name> | neo4j | Auth username. |
--auth-password <p> | "" | Shared password. Empty = no auth, fail-closed: the node won’t start without --insecure-disable-auth. |
--insecure-disable-auth | off | Explicitly allow running open (empty password). Trusted networks only. |
--tls-cert <path> | off | PEM cert chain. With --tls-key, serves the HTTP plane (REST + peer RPCs) over HTTPS. |
--tls-key <path> | off | PEM private key (required with --tls-cert). |
--tls-ca <path> | system roots | PEM CA peers verify each other with (self-signed / private-CA clusters). |
--require-metrics-auth | on (default) | Gate GET /metrics behind Basic auth. Default-on since series carry db=<name> labels. |
--insecure-open-metrics | off | Opt out of metrics auth — serve /metrics open on a private monitoring network. |
--query-timeout-secs <n> | 300 | Per-request budget for a REST /cypher read; overrun → 503 (deadline, not a hard cancel). 0 = off. |
--max-concurrent-requests <n> | 512 | Cap on in-flight HTTP requests; excess shed with 503. 0 = unlimited. |
--readiness-max-lag <n> | 50 | /ready apply-lag tolerance: report not ready (503 lagging) when last_log_index − last_applied exceeds this. See Day-2 operations. |
--otel-endpoint <url> (PATINADB_OTEL_ENDPOINT) | unset | Enable request/trace correlation (X-Request-Id + inbound W3C traceparent) and force structured JSON logs for OpenTelemetry-collector ingestion. See Day-2 operations. |
--allow-csv-dir <dir> | deny-all | Directory LOAD CSV FROM 'file://…' may read from. Repeatable. Unset ⇒ every LOAD CSV file read is refused. See Cypher file I/O. |
--allow-export-dir <dir> | deny-all | Directory the CSV export procs (patinadb.export.*) may write to. Repeatable. Unset ⇒ every export write is refused. |
--rbac-closed (PATINADB_RBAC_CLOSED) | off | Closed-mode RBAC: deny a non-admin any database it holds no explicit grant on. See Authentication & TLS. |
--telemetry-degrade-override-until <value> (PATINADB_TELEMETRY_DEGRADE_OVERRIDE) | unset | Time-boxed break-glass: keep writes enabled past a missed telemetry grace window until an absolute deadline. See Day-2 operations. |
Environment: PATINADB_AUTH_PASSWORD sets the auth password (preferred over a
shell flag).
YAML config file (--config)
Instead of (or alongside) flags, point the server at a YAML file mirroring the settings above:
# node.yaml
id: 1
addr: "127.0.0.1:21001"
db: "/var/lib/patinadb"
bootstrap: true
bolt_addr: "0.0.0.0:7687"
auth_user: "neo4j"
# auth_password: prefer the PATINADB_AUTH_PASSWORD env var over the file
query_timeout_secs: 30
max_concurrent_requests: 256
patinadb-raft --config node.yaml
Every field is optional; an absent key falls back to the CLI flag, then to the
built-in default. The YAML key names match the long flag names with -
replaced by _ (e.g. --bolt-addr → bolt_addr). Unknown keys are rejected so
a typo fails loudly.
Precedence (highest wins): an explicitly-passed CLI flag (or its bound env
var, e.g. PATINADB_AUTH_PASSWORD) > the config file > the built-in
default. A flag left at its clap default does not override a value set in the
file — only flags the operator actually typed do. The required settings (id,
addr, db) may come from either the file or flags; if neither supplies one,
startup fails with a clear error.
Config JSON Schema (--print-config-schema)
patinadb-raft --print-config-schema prints a JSON Schema (Draft 7) for the
config file — every property carries its description (lifted from the Rust
doc-comments) so editors can offer autocompletion and validation. It works
without any other arguments:
patinadb-raft --print-config-schema > patinadb-config.schema.json
Resource limits & quotas
Guards that bound the cost of a single query so one statement can’t exhaust
memory or run unbounded. The two engine budgets are environment variables
(they apply to the embedded library and every server built on it); the two
server request limits are flags (see the patinadb-raft table above).
| Limit | Where | Default | Purpose |
|---|---|---|---|
PATINADB_MAX_HOPS | env | 1000 | Depth cap for an unbounded variable-length hop ([*], or [*..n] with n unset). An explicit [*a..b] in the query always wins. Prevents runaway traversal on a cyclic graph. |
PATINADB_CARTESIAN_CAP | env | 10000 | Max rows a disjoint (cross-product) multi-MATCH may produce before the query errors. Stops an accidental N×M blow-up. |
PATINADB_MAX_AGG_ROWS | env | 5000000 | Cap on the O(input)/O(result) row buffers behind GROUP BY/aggregate, a full (non-top-K) ORDER BY, UNION dedup, and a hash-join build side. A clear error over the cap instead of a silent OOM. See Query Planning. |
PATINADB_MAX_CAPTURE_OPS | env | 5000000 | Cap on the number of resolved ops a single embedded write statement (or the Raft leader’s resolve step) may buffer before recording one engram/Raft entry. A whole-graph SET/bulk CREATE over the cap fails with a clear error pointing at CALL {…} IN TRANSACTIONS instead of risking an OOM or a giant single Raft entry. |
PATINADB_MAX_ALGO_WORK | env | ~1e10 | Static work-budget backstop for O(V·E) read procedures (betweenness/closeness): refuses a call whose estimated n·(n+e) exceeds this, so a Reader can’t trigger unbounded compute even without --query-timeout-secs. See Day-2 operations. |
PATINADB_MAX_SNAPSHOTS | env | unlimited (0/unset) | Prunes on-disk periodic time-travel snapshot files down to the N most recent (+ every pinned/tagged one) after each snapshot-taking commit. Embedded/CLI have no other retention driver, so a long-lived write-heavy embedded db otherwise grows snapshot files unbounded; pruning only slows reconstruction of an old, out-of-window engram — every AS OF result stays byte-identical. Also settable via Dataset::with_max_snapshots in the embedded API. |
--query-timeout-secs | server flag | 300 | Per-request wall-clock budget for a REST read; overrun → 503. 0 = off. |
--max-concurrent-requests | server flag | 512 | Max in-flight HTTP requests; excess shed with 503. 0 = unlimited. |
Set an env budget to 0 (or unset) to fall back to the built-in default. Tighten
them as DoS guards on a shared node, or raise PATINADB_MAX_HOPS for a
genuinely deep graph. A RETURN … LIMIT k is the normal way to bound result
size — there is no implicit result cap (an unlimited query streams every row).
For observability, set PATINADB_SLOW_QUERY_MS (server env, off by default)
to log a WARN for any REST query slower than that many milliseconds — carrying
the query’s normalized shape (literals + $params folded to ?), not raw
values. Per-shape latency stats are also served at GET /mgmt/queries.
Commercial entitlements
The server resolves a set of commercial caps (cluster HA size, combined
node+edge scale, database count, history-retention window, and two feature
gates) either from the hard-coded Community ceiling or from a signed license
token’s entitlement claims. This is configured entirely by which license you
install (--license / PATINADB_LICENSE / <db-root>/license.key — see
Licensing & Telemetry), not by a server flag. See
Editions & Limits for the full Community/Pro/Enterprise table,
what each limit does when you hit it, and how to read a running node’s
resolved tier + live usage via GET /version (also scraped onto
patinadb_entitlement_usage_ratio{axis} / patinadb_entitlement_limit{axis}
Prometheus gauges).
Cache memory budget
patinaDB can keep a governed, RAM-budgeted cache of decoded objects / adjacency
/ query results above redb’s page cache. It is opt-in: with PATINADB_CACHE_LIMIT
unset (or 0) the cache is entirely off — only the OS/redb page cache and the
existing plan/stats caches are used, at zero residual cost.
The budget derives from a cgroup-aware total (the real ceiling the kernel
OOM-kills at in a container, not the host’s RAM): cgroup v2 memory.max →
v1 memory.limit_in_bytes → host MemTotal, taking min(cgroup, MemTotal).
From that total, PATINADB_MEMORY_LIMIT is patinaDB’s own-heap ceiling (not
including the OS page cache), and four regions are carved from it. Each knob is
an absolute size (8GiB), a fraction of its parent (40%), or auto;
precedence is explicit-absolute > fraction > default, and fractions compose
against the resolved parent.
| Env var | Region | Default | Parent |
|---|---|---|---|
PATINADB_MEMORY_LIMIT | own-heap ceiling (excl. page cache) | auto = TOTAL − min_free | discovered TOTAL |
PATINADB_CACHE_LIMIT | cache pool (L1/L2/L3) — unset/0 disables caching | 40% | MEMORY_LIMIT |
PATINADB_WORK_MEM_LIMIT | action reserve (concurrent query working memory) | 45% | MEMORY_LIMIT |
PATINADB_MEM_HEADROOM | transient-spike / allocator-slop / OOM safety | 15% | MEMORY_LIMIT |
PATINADB_CACHE_MIN_FREE | page-cache floor (system free RAM kept resident for redb L0) | max(1GiB, 10%) | discovered TOTAL |
Every knob is also a patinadb-raft flag (--memory-limit, --cache-limit,
--work-mem-limit, --mem-headroom, --cache-min-free) and a YAML config key
(memory_limit, cache_limit, work_mem_limit, mem_headroom,
cache_min_free), with the same explicit-flag/env > file > default
precedence as every other setting.
The budget is validated at startup and fails loud (the node refuses to boot)
when it over-commits — CACHE_LIMIT + WORK_MEM_LIMIT + HEADROOM ≤ MEMORY_LIMIT
and MEMORY_LIMIT + CACHE_MIN_FREE ≤ TOTAL — with an error naming the offending
knobs and the resolved bytes. When enabled, the fully-resolved budget (bytes per
region) is logged at startup so it is never a mystery.
Example — a container with memory.max=16GiB, PATINADB_CACHE_LIMIT=40% and
otherwise defaults: TOTAL=16GiB, min_free≈1.6GiB, MEMORY_LIMIT≈14.4GiB
(TOTAL − min_free), then Cache ≈5.8GiB · Actions ≈6.5GiB · Headroom ≈2.2GiB.
Set PATINADB_MEMORY_LIMIT=10GiB to leave more RAM to the page cache on a
read-heavy deployment.
Internal defaults (informational)
These are not user-configurable flags today, but are useful to know:
| Setting | Value |
|---|---|
| Snapshot interval | every 50 commits |
| Raft election timeout | 750–1500 ms |
| Raft heartbeat | 250 ms |
| BM25 parameters | k1 = 1.2, b = 0.75 |
| Full-text prefix/fuzzy expansion cap | 256 terms |
| Bolt streaming channel | 256 records (bounded) |
| Default database name | default |
On-disk layout
The --db directory (server) or database path (embedded) contains the redb
tables for the graph, the property/compound indexes, the engram log and
snapshots, the full-text catalog and index data, and — for the server — the
persistent Raft log and state-machine metadata. Back up the whole directory as a
unit. For a server, each database is a subdirectory of --db.
Caching & Memory Tuning
patinaDB can keep a governed, RAM-budgeted cache of decoded graph objects
above redb’s page cache. It is opt-in and off by default — with
PATINADB_CACHE_LIMIT unset (or 0) nothing is cached beyond the OS/redb page
cache and the pre-existing plan/statistics caches, at zero residual cost.
This chapter explains why the cache exists, the memory-budget model that keeps it from fighting the page cache, and how to size, observe, and tune it. For the raw knob table (env / flag / YAML) see Configuration → Cache memory budget; this chapter is the conceptual companion.
Status. All four RAM cache layers ship on
maintoday: the memory-budget governor plus L1 decoded objects (vertices), L1 property values, L2 adjacency, and L3 query results — see the Layers section — all opt-in viaPATINADB_CACHE_LIMIT(off by default, zero residual cost). A fifth, L4 disk-backed victim tier for expensive L3 results also ships, opt-in viaPATINADB_L4_VICTIM_MAX_BYTES.
Why a cache above the page cache
redb — and, beneath it, the OS — already keep hot B-tree pages resident. That
is level 0, and it is good: it avoids disk I/O. But L0 caches bytes, and it
stops there. Every read still re-decodes those bytes: bincode /
encode_for_index → Vertex / Edge / AttributeValue, with each string value
heap-allocated multiple times on the decode → hydrate → pack path. On a hot OLTP
path (point lookups, MATCH (n) WHERE n.id = …, fan-out target reads) that decode
CPU is paid again and again for the same object.
The patinaDB cache sits above the byte boundary and caches the decoded artifact, so a hit skips the decode and the string allocations entirely. It never mmaps or manages pages itself — L0 stays the I/O-avoidance layer, and the cache is strictly a read-side accelerator: writes always go straight to redb through the durable path, and the affected cache entries are invalidated, never written through. A cache hit can only make a read faster (or memory tighter), never return a stale or wrong result.
The memory budget model
The app cache is a second consumer of the same RAM redb’s page cache needs. Grown naively it trades a decode-CPU win for extra page faults — a bad trade once the working set approaches RAM. So the budget is not a fixed number; it is an explicit, cgroup-aware partition that leaves the OS page cache a guaranteed floor.
How much do we have? (total discovery)
The number everything derives from is not the host’s MemTotal. In a
container that is the host’s RAM, and trusting it is the classic Docker
OOM-kill footgun (patinaDB ships in Docker). Discovery, in order:
- cgroup limit — cgroup v2
memory.max, else v1memory.limit_in_bytes(the real ceiling the kernel OOM-kills at); - host
/proc/meminfoMemTotal(bare metal / unlimited cgroup); TOTAL = min(cgroup_limit, MemTotal).
A non-Linux host, or unreadable files, falls back conservatively (better to under-budget than to over-commit against RAM we can’t measure).
The partition and the free-floor
From TOTAL patinaDB resolves its own-heap ceiling MEMORY_LIMIT (this does
not include the page cache — that is OS-managed L0, protected separately by
the floor below), then carves four regions:
| Region | Env knob | Default | What lives here |
|---|---|---|---|
| Cache pool | PATINADB_CACHE_LIMIT | 40% of MEMORY_LIMIT | the decoded-object / adjacency / result caches |
| Action reserve | PATINADB_WORK_MEM_LIMIT | 45% of MEMORY_LIMIT | all concurrent query working memory |
| Headroom | PATINADB_MEM_HEADROOM | 15% of MEMORY_LIMIT | transient spikes, allocator slop, OOM safety |
| Page-cache floor | PATINADB_CACHE_MIN_FREE | max(1GiB, 10%) | system free RAM the governor keeps below MEMORY_LIMIT so redb’s L0 stays resident |
PATINADB_MEMORY_LIMIT itself defaults to auto = TOTAL − min_free. The
page-cache floor is expressed against system free RAM, not the internal
MEMORY_LIMIT — it is the governor’s promise to the OS, orthogonal to the
internal cache-vs-action split.
Each knob accepts an absolute size (8GiB, 512MiB), a fraction of its
parent region (40%), or auto/unset for the default. Precedence per knob is
explicit-absolute > fraction-of-parent > default, and fractions compose: a
PATINADB_CACHE_LIMIT=40% is 40% of the resolved MEMORY_LIMIT, which may itself
be a fraction of the discovered TOTAL. Every knob is also a patinadb-raft
flag (--cache-limit, --memory-limit, --work-mem-limit, --mem-headroom,
--cache-min-free) and a YAML config key, with the usual
explicit-flag/env > file > default precedence — see the
Configuration reference.
The elastic priority
Cache and query execution (“actions”) draw from the same heap and compete. Rather than a hard wall between them, the governor applies a priority with an elastic boundary:
under memory pressure: actions > cache > (both yield to) page-cache floor
A running query that needs working memory may evict cache to grow its action
pool — a completing query beats a discardable cache — but the whole of
MEMORY_LIMIT never pushes system-free RAM below PATINADB_CACHE_MIN_FREE. Two
nested guards hold at all times: an internal one (cache + actions + headroom
≤ MEMORY_LIMIT) and an external one (MEMORY_LIMIT respects the page-cache
floor).
Fail-loud validation
At startup patinaDB resolves the whole partition and refuses to boot if it over-commits — the errors name the offending knobs and the resolved byte counts:
CACHE_LIMIT + WORK_MEM_LIMIT + HEADROOM ≤ MEMORY_LIMIT(internal partition);MEMORY_LIMIT + CACHE_MIN_FREE ≤ TOTAL(leave the OS its floor).
When caching is enabled the fully-resolved budget is logged once at startup and
echoed on GET /mgmt/cache, so the sizing is never a mystery. The startup line
looks like this (from a boot with PATINADB_CACHE_LIMIT=256MiB):
cache budget: memory_limit=… · cache=256.0MiB · work_mem=… · headroom=… · min_free=…
With caching off it instead logs cache: disabled (PATINADB_CACHE_LIMIT unset or 0).
A worked sizing example
Take a container with memory.max=16GiB and the defaults. TOTAL discovers as
16GiB; MEMORY_LIMIT=auto resolves to TOTAL − min_free (the floor keeps
≥1.6GiB of system RAM free so redb’s L0 breathes); the three internal regions
then split it roughly Cache ≈ 5.8GiB · Actions ≈ 6.5GiB · Headroom ≈ 2.2GiB.
Tune from there:
- Read-heavy / point-lookup workload — leave more RAM to the page cache:
PATINADB_MEMORY_LIMIT=10GiB(≈6GiB stays with L0), and the decoded-object cache still captures the hot decode CPU. - Feed / dashboard workload with heavy repeated reads — bias toward the
cache:
PATINADB_CACHE_LIMIT=70%, where repeat-read cache value dominates.
The governor
The budget is the contract; the governor is the runtime feedback loop that enforces it, “always keeping free RAM in view”:
- Observe. A lightweight background sampler reads
MemAvailable(/proc/meminfo) on a 1-second interval — the ground truth of how much RAM is actually free right now, including pressure from other processes on the box. - Protect the page cache first. When free RAM drops below
PATINADB_CACHE_MIN_FREE, the governor shrinks the app cache by the deficit before the OS starts reclaiming the page-cache pages redb depends on. A hysteresis window keeps a steady stream of below-floor samples from re-evicting every tick (no thrash). Its standing bias is “shrink app caches first under pressure.” - Admission = scan resistance. Admission uses W-TinyLFU (a small frequency sketch): a candidate is admitted only when it is estimated hotter than the entry it would evict. So a one-shot full analytics scan streams through without evicting the hot OLTP working set — a full table scan won’t flush the cache. Eviction within the budget is segmented LRU (probation → protected), weighted by value density = (hit-frequency × cost-avoided) / bytes, so a cheap-to-recompute big object yields before an expensive small one.
- Node-local, never stale. Caches are strictly node-local — nothing
travels over Raft, and there is no cross-node coherence protocol. Correctness
comes from generation tags: every write bumps the touched labels’
write-generation once at the
sync()choke-point (the same mechanism behindfully_populatedand the statistics catalog), and every cached entry carries the generation(s) it was built under. A lookup whose stamped generation no longer matches the live one is a miss + evict, never a stale hit — so a write to a label invalidates all of that label’s cached entries in O(1), with no scan and no per-key invalidation list. Time-travel (AS OF) reads and the copy-on-read write-resolve mirror both bypass the live cache.
What gets cached (the layers)
The design is a hierarchy above the page layer, extended incrementally:
| Level | What | Status |
|---|---|---|
| L0 | OS / redb page cache (bytes) | pre-existing; avoids I/O |
| L1 objects | a vertex’s decoded label, keyed (db, uuid); a hit skips the bincode decode on point lookups and every fan-out target read | shipped (increment 1) |
| L1 properties | a vertex’s decoded property values (AttributeValue), keyed (db, uuid, prop); a hit skips the decode + string allocations on property projections, WHERE, ORDER BY, and group-by keys — the largest decode win (values dominate the allocation) | shipped (increment 2) |
| L2 adjacency | a hot anchor’s neighbour / incident-edge list per rel-type, paired with the edge-sorted index for feed pagination; eagerly invalidated on any incident-edge write (an edge write bumps no label generation, so eager invalidation — not the stamp — is the guarantee) | shipped (increment 2) |
| L3 results | materialized results for hot, pure-read, deterministic parameterized shapes (≤ 1 MiB, seen ≥ 2), stamped with every involved label’s generation so a write to any of them invalidates it; edge/traversal, all-vertices, and procedure queries are deliberately not cached (an edge write can’t be caught by a label stamp) | shipped (increment 2) |
| L4 victim (disk) | the cost-gated, disk-backed victim tier below L3-RAM: when an expensive L3 result is evicted, instead of discarding it, it is spilled to a separate redb file (<db_root>/_l4_victim/) and served from disk on a future identical read — L3-RAM → L4 → recompute. Survives a restart; validated by the same persisted (db-id-free) generation stamp. | shipped |
Also folded under the governor’s accounting are the pre-existing plan cache
and statistics / fully_populated catalogs.
The RAM layers share one budget + the generation-tag invalidation discipline;
GET /mgmt/cacheandCALL patinadb.cache.stats()report each level’s live bytes + hit rate. L4 is a disk tier with its own byte cap (not part of the RAM budget) — see below.
L4 victim cache (disk)
The L3 result cache is pure-RAM under the governor’s byte budget, and it discards
a result in two cases regardless of how expensive it was: on eviction (the
coldest entry is dropped when over cache_limit) and on admission rejection
(a fresh result that loses the scan-resistance comparison). In both cases the
expensive CPU/IO of computing the result is thrown away and the next identical
query pays full price. Meanwhile the box usually has spare disk.
The L4 victim cache catches exactly those victims: when an L3 result whose
measured compute cost exceeds a threshold is evicted, it is spilled to a separate
redb file under the database directory (<db_root>/_l4_victim/), and a future
identical read is served from disk (deserialize an already-materialized result)
instead of recomputed. The read order becomes L3-RAM → L4 victim → recompute,
and an L4 hit is promoted back into RAM.
- Provably as safe as L3-RAM. An L4 hit is validated against the same multi-label generation stamp L3-RAM uses — persisted beside the payload. A write to any involved label advances that label’s durable generation, so the stamp no longer matches and the entry is dropped on read (lazily) and recomputed. Never a stale hit.
- Survives a restart (unlike the pure-RAM tiers): the persisted key and stamp are db-id-free — db identity is the file location and validity rests on the per-label generations, which are durable in the main database. So an expensive dashboard query stays warm on disk across a node restart.
- Cost-gated. Only a result costing more than
PATINADB_L4_VICTIM_MIN_COST_MS(default 50 ms) is spilled — a cheap query’s disk round-trip would cost more than just recomputing it. - Bounded by its own cap with LRU eviction:
PATINADB_L4_VICTIM_MAX_BYTES(0= off, opt-in, the shipped default). L4 is not part of the RAM budget — it lives on disk and never competes with the OS page cache. - Off the hot path. L4 is read only on an L3-RAM miss and written only on the
(already-cold) eviction path, so no hot query takes an L4 lock. It inherits L3’s
admission analysis verbatim (edge/traversal/unlabeled/procedure/non-deterministic
shapes are never cached), is bypassed for time-travel (
AS OF) and the write resolve mirror, and is truncated onclear_graph/ snapshot install.
| ENV | Meaning | Default |
|---|---|---|
PATINADB_L4_VICTIM_MAX_BYTES | disk cap for the L4 file (0 = disabled) | 0 (off) |
PATINADB_L4_VICTIM_MIN_COST_MS | only spill victims costing more than this | 50 |
Enable it alongside the RAM cache (it catches RAM’s victims): e.g.
PATINADB_CACHE_LIMIT=4GiB PATINADB_L4_VICTIM_MAX_BYTES=8GiB. Observability
(a dedicated l4_victim metrics block + CALL patinadb.cache.stats row) ships
alongside it — see Cache Observability & Tuning.
Observability & tuning
The cache exposes the same per-scope accounting four ways — CALL patinadb.cache.stats(), GET /mgmt/cache, the patinadb_cache_* Prometheus
series (with a ready-made Grafana dashboard), and the patinadb-browser admin
Cache section with its read-flow Sankey — so an operator can see which
database, collection (label), or query shape is hot and how much RAM it holds.
The full treatment — every metric with its name on each surface, how to read each
one, the Grafana panel walk, and a symptom → knob tuning playbook — lives in its
own chapter: Cache Observability & Tuning. The
short version: a high hit ratio on growing patinadb_cache_bytes means the cache
is earning its keep (grow it); free-floor eviction with mem_available pinned at
min_free means it is starving the OS page cache (shrink it).
When to disable. Set PATINADB_CACHE_LIMIT=0 (the default). If the working
set comfortably fits RAM with the page cache alone and decode cost is already
negligible, the app cache adds accounting overhead for little gain — L0 plus the
plan/stats caches is the right baseline. Disabling is a fully supported, tested
configuration with zero residual cost.
Honest limits
- Write-heavy scopes self-limit. A label under constant write churn bumps its generation constantly, so its cached entries rarely survive to a second hit — the cache naturally declines to cache churny data (its value density collapses) and spends the budget where it pays. The win is therefore workload-shaped: strong for read-heavy / feed / dashboard traffic, neutral for write-saturated.
- The biggest failure mode is oversizing the app cache and starving the OS page cache — inducing the exact page-cache eviction and swap cliff the cache was meant to avoid. The page-cache floor and the governor’s “shrink app caches first” bias exist precisely to prevent this, and the gauges above make a misconfiguration visible rather than silent. When in doubt, size the cache conservatively and let a high hit rate justify growing it.
Cache Observability & Tuning
This chapter is the operator’s field guide to the governed cache: every metric it exposes, the four surfaces that expose them, and a symptom → knob playbook for turning what you see into a configuration change.
It is the practical companion to Caching & Memory Tuning, which explains why the cache exists and how its memory budget is partitioned. Read that first for the concepts (the budget model, the governor, the four layers); read this to watch the cache in production and size it right. The raw knob table lives in Configuration → Cache memory budget.
Nothing to see while off. The cache is opt-in (
PATINADB_CACHE_LIMITunset or0— the default). While disabled every surface below is a truthful all-zeros “just the OS/redb page cache” report, never an error, at zero residual cost.
The four surfaces
The same per-scope accounting is exposed four ways, from quickest to richest:
| Surface | Reach for it when | Detail |
|---|---|---|
CALL patinadb.cache.stats() | you are already in a query session and want a fast per-scope look | one row per (level, scope): bytes, entries, hit rate |
GET /mgmt/cache | you want the whole report as JSON (scripts, ad-hoc curl, the browser proxy) | budget + per-level + per-scope + governor + Sankey + live free RAM |
Prometheus /metrics | you want time-series, alerting, and the Grafana dashboard | the patinadb_cache_* gauge/counter set |
patinadb-browser admin → Cache | you want a live visual, including the read-flow Sankey | fill gauges, per-level bars, hot-scope table, Sankey |
All four read the same process-global governor, node-local by design — a node reports only its own cache. The cache is never replicated, so each node’s numbers stand alone (that is exactly why a per-node cache can never cause divergence — see Caching → The governor).
Metric reference
Every metric, grouped by what it tells you, with its name on each surface it
appears on. A — means that surface does not expose it. The Prometheus column is
the exact series name — a wrong name reads nothing, so these are verbatim.
Labels: {level} is the cache layer (l1.objects, l1.properties,
l2.adjacency, l3.results); {db} is the numeric database id.
Fill — how much RAM the cache holds
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Resident bytes per level (and per db) | patinadb_cache_bytes{level,db} | bytes (per scope) | levels[].bytes, total_resident_bytes | Growing bytes with a healthy hit ratio = the cache is earning its RAM |
| Resident entry count per level | patinadb_cache_entries{level} | entries (per scope) | levels[].entries | Entries × avg-entry-bytes ≈ bytes; a spike in entries with flat bytes = many small objects |
| Fill vs. the cap | patinadb_cache_utilization_ratio{level} | — | levels[].utilization, top-level utilization | Each level’s bytes / cache_limit; the levels sum to the overall fill. ~1.0 = full |
| Mean bytes per entry | — | — | levels[].avg_entry_bytes | Sizing sanity check — an L3 result row is far larger than an L1 object |
| The resolved hard cap | patinadb_cache_limit_bytes | — | budget.cache_limit | The ceiling the sum of all levels is kept under |
Hit / miss — is the cache paying off
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Cache hits (skipped a decode) | patinadb_cache_hits_total{level,db} | via hit_rate | levels[].hits, scopes[].hits | A hit is a decode + string-alloc avoided |
| Cache misses (fell through to storage) | patinadb_cache_misses_total{level,db} | via hit_rate | levels[].misses, scopes[].misses, sankey.misses | The fall-through to a real storage decode |
| Hit ratio | patinadb_cache_hit_ratio{level} | hit_rate (per scope) | levels[].hit_rate, scopes[].hit_rate | hits / (hits + misses). The single headline number per level / scope |
cache.stats reports hit_rate per (level, scope) — the finest grain, so
you can see which collection is hot. The Prometheus hit_ratio is the per-level
aggregate; /mgmt/cache carries both.
Eviction — is the cache under pressure
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Entries evicted by LRU/capacity, per level | patinadb_cache_evictions_total{level} | — | levels[].evicted_entries | Rising alongside a low hit ratio = the working set doesn’t fit |
| Bytes freed by those evictions, per level | — | — | levels[].evicted_bytes | The byte-weight of the per-level LRU churn |
| Bytes evicted to protect the page-cache floor | patinadb_cache_evicted_free_floor_bytes_total | — | governor.evicted_free_floor_bytes | The page-cache-pressure signal. Non-zero = the governor is shrinking the app cache to keep the OS’s L0 resident (see tuning) |
| Bytes evicted to enforce the hard cap | patinadb_cache_evicted_cap_bytes_total | — | governor.evicted_cap_bytes | The cache hit cache_limit with RAM to spare — you can afford a bigger cap |
The free-floor-vs-cap split is the most operationally important pair here.
Both are eviction, but they mean opposite things: cap eviction says the cache
is bounded by your PATINADB_CACHE_LIMIT (raise it if you have RAM);
free-floor eviction says the cache is bounded by the OS running low on free
RAM (the cache is starving the page cache — shrink it).
Admission — is scan-resistance working
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Candidates admitted | patinadb_cache_admissions_total | — | governor.admissions | The steady flow of newly-cached decoded artifacts |
| Candidates rejected (scan-resistance) | patinadb_cache_rejections_total | — | governor.rejections | A rejection is a one-shot scan element kept out of the hot set — healthy. Rises when a full scan streams cold keys past a warm, cap-full cache (victim-aware W-TinyLFU admission) |
Invalidation — the write-churn tax
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Entries dropped by a stale generation on lookup | patinadb_cache_gen_invalidations_total{level} | — | levels[].gen_invalidations | Lazy invalidation: a write bumped a label’s generation, so its cached entries miss on next read. High on a scope = that label is write-churny |
| Entries dropped by a proactive scope invalidation | — | — | levels[].scope_invalidations | Eager drops from clear_graph, db-drop, an edge write into L2, or the same-txn write window |
Both count the same thing from two directions — the cost of a write to cached
data. gen_invalidations is the passive, next-read discovery; scope_invalidations
is the active, up-front purge. A scope with high invalidation and a low hit ratio
is telling you the cache cannot help that data (it changes faster than it is
re-read) — expected, and not worth budget (see
Caching → Honest limits).
Budget & free RAM — the governor’s operating envelope
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| The system free-RAM floor the governor protects | patinadb_cache_min_free_bytes | — | budget.min_free | The promise to the OS: keep at least this much system RAM free for redb’s L0 |
Live system free RAM (MemAvailable) | patinadb_mem_available_bytes | — | mem_available_bytes | The ground truth the sampler watches. Hovering near min_free = pressure |
| Own-heap ceiling | — | — | budget.total | patinaDB’s own-heap limit (excludes the OS page cache) |
| Action reserve | — | — | budget.work_mem_limit | RAM reserved for concurrent query working memory |
| Headroom | — | — | budget.headroom | The transient-spike / OOM safety margin |
The pair to watch together is patinadb_mem_available_bytes against
patinadb_cache_min_free_bytes: the gap between them is the governor’s remaining
slack before it starts shrinking the app cache to defend the page cache.
L4 disk victim tier
The L4 victim cache is the optional disk tier below the RAM L3 result
cache: when an expensive L3 result is evicted (or admission-rejected), it is
spilled to a redb file under the db root and served from disk on a future
identical query instead of recomputed. It is off by default
(PATINADB_L4_VICTIM_MAX_BYTES=0), so its whole metric block is absent until
you enable it. Enabled, it reports as a distinct block — it is not a governed
RAM level, so it never shows up under levels[] or the RAM cache_limit.
| Meaning | Prometheus | cache.stats | /mgmt/cache | How to read it |
|---|---|---|---|---|
| Resident disk entries / bytes | patinadb_l4_victim_entries / patinadb_l4_victim_bytes | l4.victim row (entries/bytes) | l4_victim.entries / .bytes | Fill against the configured max_bytes cap |
| Valid disk hits (served + promoted) | patinadb_l4_victim_hits_total | l4.victim row hit_rate = hits/(hits+stale_drops) | l4_victim.hits | Each hit skipped a full recompute of an expensive query |
| Hits promoted back to RAM | patinadb_l4_victim_promotions_total | — | l4_victim.promotions | An L4 hit re-enters L3 (a proven repeat) |
| Victims spilled to disk | patinadb_l4_victim_spills_total | — | l4_victim.spills | Expensive results caught on evict/reject. spills ≫ hits = you are paying disk churn for results that never get reused — raise PATINADB_L4_VICTIM_MIN_COST_MS or lower the cap |
| Entries dropped stale on read | patinadb_l4_victim_stale_drops_total | — | l4_victim.stale_drops | The persisted generation stamp mismatched (a write touched an involved label) — high = the cached labels are write-churny (L4 can’t help them) |
| Entries LRU-evicted over the disk cap | patinadb_l4_victim_evictions_total | — | l4_victim.lru_evictions | The disk tier is at max_bytes — the coldest entries are dropped first |
The headline pair is spills vs hits: L4 is earning its keep when hits are a
healthy fraction of spills. If spills dominate, the workload is expensive-but-not-
repeated (or too write-churny — see stale_drops), and the disk tier is pure
overhead. The writer runs on a bounded background channel off the read/evict
hot path, so a slow disk never stalls a query — under backpressure a spill is
simply dropped (a future recompute, never a wrong answer).
Reading each surface
In-query: CALL patinadb.cache.stats()
The fastest look — no HTTP, no dashboard, runs in any session:
CALL patinadb.cache.stats()
YIELD scope, kind, bytes, entries, hit_rate, generation
RETURN scope, kind, bytes, entries, hit_rate
ORDER BY bytes DESC
scope— the hot database / collection / query shape, rendered as a stable string:db:1,db:1/label:Ticket, ordb:1/shape:1234(a plan fingerprint).kind— the cache level (l1.objects,l1.properties,l2.adjacency,l3.results, and — when the disk victim tier is enabled — a singlel4.victimpseudo-scope row).bytes/entries— resident size for that(level, scope).hit_rate—hits / (hits + misses),0.0when never accessed.generation— reserved; yielded asNULLtoday (the column is kept for schema stability).
With caching disabled it returns zero rows (no levels are registered), never an error. See Procedures → Cache observability.
Over HTTP: GET /mgmt/cache
The complete report as one JSON document (admin-only — it is under /mgmt/). This
is the richest single call: it carries fields no other surface has
(avg_entry_bytes, evicted_bytes, scope_invalidations, the full budget, and
the Sankey).
curl -s -u neo4j:secret http://127.0.0.1:21001/mgmt/cache | jq
An enabled node returns roughly:
{
"enabled": true,
"budget": {
"total": 15461882265,
"cache_limit": 6184752906,
"work_mem_limit": 6957846769,
"headroom": 2319282256,
"min_free": 1717986918
},
"total_resident_bytes": 41231360,
"utilization": 0.0067,
"governor": {
"admissions": 128934,
"rejections": 20514,
"evicted_free_floor_bytes": 0,
"evicted_cap_bytes": 0
},
"levels": [
{
"name": "l3.results",
"bytes": 12058624, "entries": 214,
"hits": 90233, "misses": 1201, "hit_rate": 0.9869,
"utilization": 0.0019, "avg_entry_bytes": 56348,
"evicted_entries": 0, "evicted_bytes": 0,
"gen_invalidations": 88, "scope_invalidations": 3,
"scopes": [
{ "scope": "db:1/shape:8123", "db": 1, "bytes": 8388608,
"entries": 40, "hits": 61022, "misses": 210, "hit_rate": 0.9966 }
]
}
],
"sankey": {
"total_lookups": 402118,
"misses": 14002,
"layers": [
{ "name": "l3.results", "label": "L3 result", "value": 90233 },
{ "name": "l2.adjacency", "label": "L2 adjacency", "value": 121444 },
{ "name": "l1.properties","label": "L1 property", "value": 130221 },
{ "name": "l1.objects", "label": "L1 object", "value": 46218 },
{ "name": "miss", "label": "miss → storage","value": 14002 }
]
},
"mem_available_bytes": 9663676416
}
The levels array is ordered deepest-cache-first (L3 → L2 → L1 property → L1
object) — the same order the Sankey reads. Each level’s scopes are sorted
hottest-first (by hits, then bytes) so the top row is the hottest collection or
shape. The sankey block is the read-flow: total_lookups is the source width
(Σ hits + Σ misses), each layer’s value is the hits that layer absorbed,
and the trailing miss layer is the summed fall-through to a storage decode.
Because the levels are consulted independently (a query may touch several), it is
an aggregate share of all cache lookups, not a strict per-query cascade.
While disabled, enabled is false, the budget is all zeros, and levels is
empty. See REST API → GET /mgmt/cache.
Prometheus & Grafana
/metrics exports the full patinadb_cache_* set (refreshed from the governor at
scrape time — always current, no background task). The Docker demo in deploy/
ships a ready-made patinaDB Cache dashboard
(grafana/dashboards/patinadb-cache.json, 20 panels); it is auto-provisioned by
the compose stack, so once a node is scraped it appears in Grafana with no import
step. To load it into an existing Grafana, import that JSON and point it at your
Prometheus datasource. Its three panel rows map onto the metric groups above:
- Cache Overview — five stat tiles: utilization %
(
sum(patinadb_cache_bytes) / max(patinadb_cache_limit_bytes)), resident bytes, cached entries, mem-available, and the overall hit ratio (rate(hits) / (rate(hits) + rate(misses))). - Fill & Hit Rate — resident bytes / utilization / hit-ratio / entries per
level, the hit-vs-miss rate, and a hot databases table keyed by
patinadb_cache_bytes{level,db}. - Eviction, Admission & Invalidation — the free-floor-vs-cap eviction-byte
split (
rate(patinadb_cache_evicted_free_floor_bytes_total)vs..._cap_bytes_total), evicted entries and generation invalidations per level, admissions vs rejections, the miss → storage decode rate, and mem-available vs the free floor — the single most important page-cache-pressure panel.
The dashboard is all-zero until the cache is enabled. The full compose stack and
its two provisioned dashboards are documented in the repo’s deploy/README.md.
Browser admin → Cache
The patinadb-browser admin dashboard has a Cache section (it proxies the
node’s GET /mgmt/cache) with, at a glance:
-
KPI tiles — resident / cache-limit and utilization, governor admissions & rejections, the floor-vs-cap eviction bytes, and live mem-available vs the free floor.
-
Per-level bars — one row per level with a hit/miss track and the raw hits / misses / evictions / gen-invalidations / utilization.
-
Hot scopes — the top scopes across all levels (which db / collection holds the hot bytes).
-
The Layer Sankey — “Which layer catches how much”: one ribbon per cache layer sized by the hits it absorbed, plus a
miss → storagefall-through:Cache lookups ─┬─▶ L3 result ├─▶ L2 adjacency ├─▶ L1 property ├─▶ L1 object └─▶ miss → storageA fat
L1 propertyribbon and a thinmiss → storagetail is the healthy read-heavy picture — most reads are absorbed above the storage decode. A fatmiss → storageribbon means the cache is not catching your read pattern (too small, or the workload is write-churny / scan-heavy). In embedded (file) mode there is no server, so the section shows cache disabled.
Tuning from the metrics
This is the payoff: mapping what a surface shows to the knob that fixes it. Every
knob below is documented in
Configuration → Cache memory budget
(each is an env var, a patinadb-raft flag, and a YAML key).
| What you see | What it means | What to do |
|---|---|---|
Low hit ratio + rising evictions_total | The working set is bigger than the cache — entries are evicted before their second hit | Raise PATINADB_CACHE_LIMIT (you have RAM to spend) |
Non-zero evicted_free_floor_bytes_total + mem_available hovering near min_free | The app cache is starving the OS page cache; the governor is shrinking it to defend redb’s L0 | Lower PATINADB_CACHE_LIMIT (or PATINADB_MEMORY_LIMIT), or raise PATINADB_CACHE_MIN_FREE to give L0 a bigger floor |
Rising evicted_cap_bytes_total while mem_available stays healthy | The cache is bounded by your cap, not by RAM pressure — there is free RAM going unused | Raise PATINADB_CACHE_LIMIT to let the cache grow into the free RAM |
High gen_invalidations / scope_invalidations on a scope + low hit_rate there | That label is write-churny; its entries die before a second read | Expected — nothing to tune. The cache correctly declines to spend budget on it; don’t force it |
Rising rejections_total | Admission (scan-resistance) is keeping a one-shot scan out of the hot set | Healthy — no action. This is the cache protecting your OLTP working set from an analytics scan |
Near-100% utilization + high hit_ratio + low eviction rate | Well-sized: the cache is full of hot data and rarely churns | Leave it. Grow the cap only if the hit ratio starts to dip |
Fat miss → storage Sankey ribbon | Most reads fall through to a storage decode | Cache too small (raise the limit) or the workload is genuinely write-/scan-heavy (accept it, or see When to disable) |
The two failure modes worth internalizing are the mirror image of each other:
- Too small shows as low hit ratio + cap eviction + a fat miss ribbon while
RAM is free → raise
PATINADB_CACHE_LIMIT. - Too big shows as free-floor eviction +
mem_availablepinned atmin_free→ lower it. Oversizing the app cache and starving the OS page cache re-creates the exact swap cliff the cache was meant to avoid — the free-floor split and the mem-available panel exist to make that visible before it bites.
When in doubt, size conservatively and let a high hit ratio justify growing.
If the working set fits RAM with the page cache alone and decode cost is already
negligible, the honest answer is PATINADB_CACHE_LIMIT=0 (the default) — a fully
supported, zero-residual-cost configuration; see
Caching → When to disable.
Current limitations
generationincache.statsis reserved — the per-scope write-generation is not carried on the observability seam yet, so the column is alwaysNULL. It is kept in the signature for schema stability.- Prometheus is a subset of
/mgmt/cache.avg_entry_bytes,evicted_bytes(per level), andscope_invalidationsare only on/mgmt/cache(and the browser) — there is no Prometheus series for them. For alerting on those, scrape the endpoint directly.
Editions & Limits
patinaDB’s server (patinadb-raft) ships as one binary with three commercial
editions gated by a signed license token. This chapter is the
honest, public table of what each edition includes, what each limit
actually does when you hit it, and how to watch your own usage before you do.
The guiding principle: the wall is for the successful, never the evaluator. Community is deliberately generous — a real evening project, or even a medium-sized proof-of-concept, should never brush against a cap. The limits start to matter once a deployment is running real production traffic: needs failover, needs more than a couple of databases, or needs to keep the whole audit history forever.
The table
| Axis | Community | Pro | Enterprise | What it’s for |
|---|---|---|---|---|
| Cluster voters (HA) | 1 (no failover) | 5 | unlimited | A single voter has no automatic failover — production reliability needs more than one, which is the primary commercial wall. |
| Nodes + edges (combined) | 5,000,000 | 100,000,000 | unlimited | A backstop, not the main fence — generous enough that a real medium-graph evaluation never hits it. |
| Databases | 2 | 20 | unlimited | Multi-tenant / multi-workload isolation is a business feature. |
| History retention | 30 days rolling | unlimited | unlimited | Time-travel itself is free in every edition (see below) — keeping the entire timeline forever, for compliance/audit, is the Pro/Enterprise sell. |
| Fine-grained (per-label) security | off | on | on | GRANT/REVOKE READ|WRITE ON <db>:<Label> + the /mgmt/audit security log. |
| Point-in-time-recovery backup | off | on | on | GET /mgmt/snapshot?history=true — a portable backup that carries the whole engram timeline, not just HEAD state. |
| Telemetry | mandatory (degrades if unreachable) | best-effort, --disable-telemetry | best-effort, --disable-telemetry / fully air-gapped | See Licensing & Telemetry. |
Every other capability — full Cypher + Bolt, graph algorithms, full-text and vector search, spatial queries, time-travel itself, RBAC with blanket per-database roles, engrams/diffs, change streams, everything in this manual that isn’t in the table above — is identical across every edition. The limits above are the entire commercial fence; nothing else is gated.
What happens when you hit a limit
Two different things happen depending on which axis you hit, and neither one ever touches data that’s already there:
- The scale cap (nodes + edges) degrades to read-only. Once a database’s
combined node+edge count would cross the cap, further writes are
refused with a clear, actionable error (
503over REST, a Bolt failure) — your data stays exactly as it was, and every read keeps working normally. ADELETEthat shrinks the graph back under the cap is still allowed, so you can always recover headroom without needing to buy anything. This is the same mechanism the community telemetry gate uses (see Licensing & Telemetry) — a write choke-point that either lets a proposal through or refuses it with an upgrade message. - The cluster/database/feature caps refuse the specific operation. Trying
to promote a
(max_voters + 1)-th voter, create a(max_databases + 1)-th database, or run a feature-gated command (a per-labelGRANT,/mgmt/audit,?history=true) past Community’s ceiling fails loudly with an upgrade message — but the running system is completely untouched. Nothing you already have breaks; the specific action you tried just doesn’t happen.
History retention in practice
Community’s 30-day rolling retention window means: AS OF <engram> /
CALL patinadb.diff(<engram>) work for anything committed in roughly the last
30 days. Older history is periodically compacted (squashed into a single
snapshot at the retention boundary) rather than kept forever — this shrinks
the engram log, not the live graph, so HEAD data and every current query are
completely unaffected. A Pro or Enterprise license simply omits the retention
cap, so nothing is ever compacted and the full timeline (all the way back to
the very first commit) stays queryable.
Reading your own usage
GET /version (no authentication required, same as /health) reports the
resolved tier, every entitlement cap, and a live usage snapshot — so you can
see headroom before hitting a wall instead of finding out from a failed
write:
curl http://localhost:8080/version
{
"name": "patinadb-raft",
"version": "0.9.0",
"tier": "community",
"entitlements": {
"max_elements": 5000000,
"max_voters": 1,
"max_databases": 2,
"history_retention_days": 30,
"fine_grained_security": false,
"pitr_backup": false
},
"usage": {
"nodes": 812345,
"edges": 1204981,
"elements": 2017326,
"elements_usage_ratio": 0.403
}
}
elements_usage_ratio is null whenever the cap is unlimited (a licensed
Enterprise deployment, or any axis a license simply omits) — there is nothing
to divide against, so no ratio is ever emitted for an uncapped axis. Watch this
value (or scrape it — see below) and you’ll see the “you’re at 40% of the
community node limit” signal well before you ever hit the wall.
Prometheus
The same usage figures are exported as gauges on the existing /metrics
scrape endpoint:
patinadb_entitlement_usage_ratio{axis="elements"}— current / cap, current / cap, emitted only when theelementscap is finite.patinadb_entitlement_limit{axis="elements"|"voters"|"databases"}— the resolved numeric caps themselves, so a dashboard panel can render headroom without re-deriving it from the license.
Wire these into the same Grafana dashboard as the rest of the cluster metrics (see Configuration Reference and Cache Observability & Tuning for the sibling observability surfaces) to get an early warning before a busy database approaches its scale cap.
Getting a license
A license carries signed entitlement claims — the specific numeric caps and feature flags a tier unlocks (an Enterprise license simply omits every numeric cap, so every axis resolves to unlimited). Licenses are issued by patinaDB; contact your vendor or account representative to obtain one for your tier.
See Licensing & Telemetry for how licenses are verified and
installed once you have one. A node with no license (or an invalid/expired
one) always resolves to the hard-coded Community ceiling in the table above.
Check what a running node actually resolved to via GET /version (see
Reading your own usage).
Licensing & Telemetry
patinaDB’s server (patinadb-raft) runs in one of two modes. The distributed
Docker image ships without a license, so by default a node runs in
community mode and sends an anonymous usage heartbeat to a telemetry server.
Installing a license file switches the node to licensed mode for
on-prem / offline / air-gapped operation, where telemetry is best-effort and can
be turned off entirely.
This chapter explains both modes, documents exactly what the telemetry heartbeat contains (and, just as importantly, what it never contains), and shows how to obtain and install a license.
The two modes
Community mode (default — no valid license)
A background heartbeat is mandatory. On startup the node sends an initial heartbeat, then repeats it on an interval. It tolerates transient network outages (retries with backoff, plus a long grace window), so a brief blip is harmless. But it must not run indefinitely offline:
- If no heartbeat succeeds within the grace window (default 72 hours),
the node degrades. It refuses client writes with a clear
503error and logs the reason loudly. - Reads keep working during and after the grace window — degradation only blocks writes.
- Control / admin commands (creating databases, users, indexes) are not blocked, so an operator can still recover the node.
- As soon as a heartbeat succeeds again, the block is lifted automatically and writes resume.
The refusal is surfaced as an HTTP 503 (REST) or a Bolt failure, with a
message explaining that the node could not reach the telemetry server and is
running unlicensed.
Licensed mode (valid license file)
A valid license unlocks on-prem operation:
- Telemetry is best-effort: it is still sent by default (so the maintainer can see version adoption), but a failure to send never blocks anything and the node never degrades.
- Telemetry can be turned off completely with
--disable-telemetry. --disable-telemetryis honored only with a valid license. An unlicensed node started with--disable-telemetryrefuses to start (fail-closed) with a clear message — a community node must send telemetry.
What the telemetry heartbeat sends
The heartbeat is a coarse, anonymous JSON POST. The payload is a strict
allowlist of counters and environment facts. Nothing outside the table
below is ever sent. This is not a promise on paper alone: an automated guard
test asserts the serialized payload’s key set equals this allowlist, and a
second doc↔code drift test parses this very table and asserts it lists exactly
the fields the code sends — so if a field is ever added to the wire without
being documented here (or vice versa), the build fails. The table cannot
silently drift from reality.
Every field, exhaustively:
| Field | Type | Meaning |
|---|---|---|
install_id | UUID | A random id generated once and persisted at <db-root>/.patinadb_install_id. Stable and anonymous — not derived from anything identifying. |
nonce | UUID | A fresh random value generated per heartbeat. It exists only so the server’s signed response can be bound to this exact request (anti-tamper — see Response signing); it carries no information about you. |
install_name | string | null | Opt-in, operator-chosen label (--install-name / PATINADB_INSTALL_NAME). Unset ⇒ null (anonymous), the default. This is the only free-text field and it is consent-based: it is whatever you type, and is never auto-derived from the environment (no hostname, no username). |
version | string | The node build version. |
protocol_version | integer | The Raft protocol version. |
os | string | Target OS (e.g. linux). |
arch | string | Target CPU architecture (e.g. x86_64). |
cores | integer | Logical CPU core count (≥ 1). A coarse hardware-sizing signal. |
memory_bytes | integer | Total host/cgroup RAM in bytes. A coarse hardware-sizing signal — not a live memory-usage figure. |
uptime_secs | integer | Seconds since this process started. |
node_count | integer | Cluster membership size (voters + learners). |
is_leader | boolean | Whether this node is currently the Raft leader. |
database_count | integer | Number of databases in the registry. |
total_vertices | integer | Aggregate vertex count across all databases (a coarse volume signal only). |
total_edges | integer | Aggregate edge count across all databases (a coarse volume signal only). |
requests_per_min | number | Coarse recent request rate: total recorded query executions ÷ uptime minutes. An aggregate count only — carries no query text. |
avg_query_ms | number | Mean query latency in milliseconds, aggregated (count-weighted) across all query shapes. An aggregate timing only — carries no query text. |
license_status | string | "community" or "licensed". |
license_customer | string | Licensed mode only. The licensee’s own customer id from the license (the buyer of the license — this is not an end-user). Omitted in community mode. |
What we NEVER send
The telemetry payload will never contain any of the following. When in doubt, it is left out:
- No database content — no rows, no graph data of any kind.
- No names — no property keys or values, no label or relationship-type names, and no database names.
- No queries — no query text or any fragment of one (the
requests_per_minandavg_query_msfigures are pure aggregate counters/timings). - No node or edge UUIDs.
- No hostname, IP address, username, or location. (
install_nameis the one label you may choose to send — it is never read from the host.) - No RBAC user names, engram messages, or authors.
- No personally-identifiable information of any kind.
Only the coarse counts and environment metadata in the table above leave the node.
How and when it sends; the endpoint
On startup the node sends one heartbeat, then repeats it on the interval. It is a
single coarse anonymous POST — see what it sends
above. The stable, anonymous install_id lets the maintainer count distinct
installs across restarts without knowing anything about who you are; the optional
install_name is a label you may choose to attach so your own installs are
recognizable in your reports — it defaults to anonymous and is never taken from
the host.
The endpoint is a compile-time constant baked into the binary — for the
distributed community build, the maintainer’s server. In a release build it is
NOT overridable: the --telemetry-endpoint flag and the
PATINADB_TELEMETRY_ENDPOINT environment variable are compiled out, so a
community node always phones the real host (redirecting it to /dev/null would
defeat the point of community telemetry). Only debug/test builds accept an
override, so the test suite can point at a mock server.
Honest note on the endpoint lock. Locking the endpoint is a soft deterrent, not a hard control. Telemetry is best-effort, so a determined operator can still firewall-block the host or patch the binary — we don’t pretend otherwise. The lock also means a licensed organization cannot point the heartbeat at its own telemetry server; the supported path for a licensed org that doesn’t want to phone home is
--disable-telemetry(turn it off), not redirection.
| Setting | Flag | Env var | Config key | Default |
|---|---|---|---|---|
| Heartbeat interval | --telemetry-interval-secs <n> | — | telemetry_interval_secs | 21600 (6h) |
| Grace window | --telemetry-grace-secs <n> | — | telemetry_grace_secs | 259200 (72h) |
| Opt-in install label | --install-name <text> | PATINADB_INSTALL_NAME | install_name | unset (anonymous) |
| Disable telemetry (licensed only) | --disable-telemetry | — | disable_telemetry | false |
The endpoint itself has no run-time flag in release (see above). See the Configuration Reference for how flags, environment variables, and the config file combine.
Response signing (anti-tamper server authentication)
Locking the endpoint stops a community node from redirecting its phone-home,
but a determined operator could still point telemetry.patinadb.org at a
fake server (via /etc/hosts or DNS) that just returns 200 OK — dodging
the degrade-after-grace without ever reaching the real host. To close that,
the real telemetry server cryptographically signs its ping response and the
node verifies the signature.
How it works:
- The node sends a fresh random
noncewith each heartbeat (see the field table above), alongside itsinstall_id. - The server signs its response. The genuine server holds an Ed25519
private key and returns
{ server_time, signature }, where the signature coversnonce ‖ install_id ‖ server_time. Only a server holding that private key can produce a signature the node accepts. - The node verifies the signature against a second Ed25519 public key
embedded in the binary (
TELEMETRY_RESPONSE_PUBLIC_KEY, separate from the license key), checks thenoncematches the one it sent, and checksserver_timeis fresh (within ±5 minutes, to bound replay). - The verdict feeds the grace clock. A valid signature is a successful heartbeat (it resets the grace clock). A missing, malformed, forged, or stale signature is a failed heartbeat — it does not reset the clock, so a community node still degrades after the grace window.
A fake server has no private key, so it cannot produce a valid signature and
therefore cannot stop the node from degrading. Because the signing key lives
entirely outside the TLS / system-trust chain, this defeats even an attacker who
has managed to insert a rogue CA into the host’s trust store (a plain
/etc/hosts redirect is already stopped by strict TLS certificate validation —
the node’s HTTPS client never disables cert/hostname verification).
Honest ceiling. This is an anti-forgery control, not an anti-patch control. A determined operator can still edit the embedded public key (or delete the verification) out of their own rebuilt binary, or simply firewall-block the host. Signing raises the bar from “edit one line in
/etc/hosts” to “patch and recompile the binary” — a soft deterrent consistent with the community-telemetry model. The supported opt-out for a licensed org remains--disable-telemetry, not evasion.
Opting out
There is exactly one supported way to stop telemetry, and it requires a license:
--disable-telemetry, honored only with a valid license. It turns the heartbeat off completely — nothing is sent. An unlicensed node started with this flag refuses to start (fail-closed): a community node must send telemetry.
If you run community mode and simply stop reaching the endpoint (network
block, air-gap), the node does not silently continue forever — after the
grace window (default 72h) it degrades: client writes are refused with a
503, while reads keep working, until a heartbeat succeeds again. This
“degrade after grace” behavior is the honest community-mode contract; the way to
run offline without degrading is to install a license.
Installing a license (on-prem / offline / air-gapped)
A license is a compact, Ed25519-signed token. Because the node verifies it against a public key embedded in the binary, a license can be validated with zero network access — ideal for air-gapped deployments.
A node looks for a license in this order:
- The
--license <value>flag or thePATINADB_LICENSEenvironment variable. The value may be either a path to a license file or an inline token. - A
license.keyfile in the database root (<db-root>/license.key).
If no valid license is found, the node logs why and falls back to community mode. An invalid signature or an expired license is likewise logged and treated as community mode — the node still boots, just unlicensed.
To operate a node fully offline:
- Obtain a license token (a
license.keyfile) from patinaDB, your vendor, or your account representative — see Editions & Limits for what each tier includes. - Save it as
license.keyin the node’s database root (or point--licenseat it, or setPATINADB_LICENSE). - Start the node. It logs
licensed mode (on-prem)on success. - Optionally add
--disable-telemetryto stop all outbound heartbeats.
Privacy stance
patinaDB’s telemetry is designed to be anonymous by construction: the payload is a fixed allowlist of coarse counters and environment facts, guarded by an automated test that fails the build if any field outside the allowlist is added. No database content, no names, and nothing personally identifiable ever leaves the node. Licensed / air-gapped deployments can disable telemetry entirely.
Telemetry and licensing are node-local — each node phones home independently and they are never routed through the Raft log, so they add no cross-node coordination. For authentication and transport security of the data plane, see Authentication & TLS.
Limitations
This chapter is the honest inventory of what patinaDB does not do, or does differently from Neo4j. Read it before committing to a production workload.
Query language
-
~4% of the openCypher TCK does not pass (~3712 / 3868 scenarios; ~156 failures). The TCK is a prioritized gap report, not a guarantee. The failures are not separate bugs — they cluster as follows:
Cluster ~ fails What it is User-defined procedure fixtures 50 Scenarios that register a test procedure ( there exists a procedure …). patinaDB has no user-defined procedures, so these can’t run — one limitation, 26% of the failures.Missing typed errors 34 Negative tests expecting a specific error ( InvalidArgumentType,IntegerOverflow,DeletedEntityAccess,MergeReadOwnWrites, …). patinaDB is more permissive and doesn’t raise them — a validation gap, not wrong results on valid queries.Quantifiers / comprehensions over entity lists ~14 any/all/none/singleand list comprehensions whose list holds nodes/relationships (not scalars), plus statically-true-predicate edge cases.ORDER BYedge cases~10 Ordering by expression / aggregate / cross-type value ordering. Temporal edge cases ~8 9-digit extended-year date parsing, datetimetimezone serialization,duration.betweenover huge spans.Aggregation grouping ~3 Aggregates inside non-aggregate expressions; multiple aggregates on one variable. Long tail (lists, literals, precedence, …) ~30 Many features with 1–3 scenarios each; no further large cluster. OPTIONAL MATCH … WHERE was closed (2026-07-08): a per-clause
WHEREon anOPTIONAL MATCHnow filters the optional side before the left-join null-extension (a non-matching predicate yields NULLs, not a dropped row), and a chainedOPTIONAL MATCHbinds variables introduced by an earlier optional. A negative sub-second-only ISO duration (P1DT-0.001S) also round-trips correctly now.MERGE
ON CREATE/ON MATCHis supported:SET x = <entity>,SET x += <map/entity>(nodes + relationships),SET x:Label, null-in-map removal, and combined ON CREATE+ON MATCH all work, plus MERGE matches-or- creates per row and undirected MERGE matches either direction.Known non-determinism (to fix):
GROUP BY … ORDER BY <aggregate> LIMIT napplies the limit in group-insertion order rather than after sorting by the aggregate, so the surviving rows can vary run to run. Read-only today (it does not diverge Raft followers, which apply leader-resolved ops), but a real correctness bug. -
No user-defined procedures or functions.
CALLreaches only the built-in procedures (Procedures). You cannot register your own. (This alone accounts for ~50 TCK failures, as above.) -
Constraints: uniqueness, existence (NOT NULL), node-key, property-type, and relationship existence/property-type are supported; relationship key/uniqueness is not.
CREATE CONSTRAINT [name] [IF NOT EXISTS] FOR (n:Label) REQUIRE n.prop IS UNIQUE(plus the Neo4j-4ON (n:Label) ASSERT …form and the shorthandON :Label(prop)),REQUIRE n.prop IS NOT NULL(existence),REQUIRE (n.p1, n.p2) IS NODE KEY(composite unique + existence over the key),REQUIRE n.prop IS :: <TYPE>/IS TYPED <TYPE>(property-type), and the relationship formsFOR ()-[r:TYPE]-() REQUIRE r.prop IS NOT NULL | IS :: <TYPE>are supported, along withDROP CONSTRAINT name [IF EXISTS]andSHOW CONSTRAINTS; all replicate across a cluster and are carried in Raft snapshots + portable backups. Enforcement is at write time onCREATE,MERGE, andSET/REMOVEfor nodes and on edge create/SET/REMOVEfor relationships (a UNIQUE NULL is exempt; existence rejects a missing/null required property, including aSET n.p = nullorREMOVE n.p; property-type rejects a present wrong-typed value but allows a missing/null one);CREATE CONSTRAINTrefuses to run if the existing data already violates the constraint. Node-key uniqueness is served by an automatically registered backing compound index. Not supported: relationship key/uniqueness (a clear “not supported yet” error),LIST<T>element-type refinement (bareLISTonly), and bare composite-UNIQUE without existence. See Constraints. (Index DDL —CREATE INDEX ON :Label(prop)and compound:Label(p1, p2)— is supported and replicates across a cluster; see the Data Model.) -
CREATE EDGE SORTED INDEXis not carried in Raft snapshots. The DDL replicates via the Raft log (re-run per node), so all live nodes serve it, but a node that bootstraps purely from a streamed snapshot after the log is purged will lack the def and fall back to traverse + sort until the DDL is re-issued. (Correctness is unaffected — the fallback returns identical rows.) -
No triggers, no stored procedures, no server-side scripting.
-
Side-effect counters are not tracked — query summaries don’t report
nodes created,relationships deleted, etc. TCK side-effect assertions are accepted as no-ops. -
shortestPathis supported, but the broader weighted/all-shortest-paths and full APOC-style procedure library are not. -
Some advanced predicate forms (
ALL/ANY/NONE/SINGLElist predicates,COUNT { … }subqueries, pattern comprehensions) are evaluated per-row (O(N)) — not yet lowered into the streaming operator tree for index/hash-join shortcuts — and the quantifiers do not yet fully handle lists whose elements are nodes/relationships (see the TCK table above).
Full-text search
- String properties only (matching Neo4j). No numeric/range terms inside the full-text query string.
- No phrase slop / proximity —
"a b"matches only exactly-consecutive terms;"a b"~3is not supported. - No highlighting or snippet extraction.
- Prefix/fuzzy expansion is capped at 256 terms per term to bound cost; very broad prefixes silently match only the first 256 dictionary expansions.
- Postings are updated read-modify-write with no segment merging. This is fine for typical write rates but is not engineered for very high write throughput over a large indexed corpus.
Storage & scale
- Single-machine storage. Each node holds the full graph; there is no horizontal sharding of one graph across machines. The cluster replicates, it does not partition.
- The whole graph is on one redb file per database. patinaDB targets embedded and small-to-medium graphs, not multi-terabyte datasets.
- Time-travel speed (not memory) grows with delta distance. Snapshots are captured and reconstructed as a streamed O(chunk) record run — building a periodic snapshot and rebuilding a past state both use bounded memory (the reconstruction runs into an on-disk temp store, not RAM), so the graph is not capped by memory. What still grows is time: reconstructing a point far from the nearest snapshot replays a longer delta chain, so frequent snapshots keep far-back queries fast.
- A plain
LOAD CSV … CREATEbuffers into one transaction. The CSV source streams row by row, but a bare (un-chunked) write buffers all resolved ops into a single transaction (on the server, one Raft entry) before committing — so a very largeLOAD CSV … CREATEhas anO(rows)memory cliff. Wrap it inCALL { … } IN TRANSACTIONS [OF n ROWS]to chunk it into many small commits withO(n)memory — one engram per chunk, and on a cluster oneclient_writeper chunk (both the embedded and the server/Raft paths are implemented). For millions of rows the offlinepatinadb <db> importbulk-loader is faster still.LOAD CSV … RETURN row(read) streams end-to-end. See Bulk Loading & Import. - Some sort paths are O(N). Single-key
ORDER BYuses an index fast-path, but the general executor still hasO(N)sort paths for cases not covered by the fast-path (e.g. an un-limited multi-key sort with no covering index) — see Query Planning. - On-disk format evolution needs a migration or a dump/reload. Each database
records an on-disk schema version. Because the storage encoding is
non-self-describing bincode, appending a variant to the end of a persisted
enum is compatible and needs nothing, but a struct field add/remove/reorder, an
enum variant insert/reorder, or a key/value byte-encoding change is breaking and
bumps the version. An older database is not unconditionally bricked on open:
if a migration step is registered for the version boundary it is upgraded in
place; if none is registered the open fails loud and points you at the
dump/reload path — export with the old build via the portable backup
(
GET /mgmt/snapshot), upgrade, then import (that backup is JSON and cross-version by construction). A database written by a newer build than the running binary is always rejected (there is no downgrade). - Cross-tree atomicity is guaranteed (redb backend). patinaDB runs on redb,
whose write transactions are natively cross-tree atomic. A commit, a
Raft-applied entry, and a snapshot install each land as an all-or-nothing
transaction of any size: a crash before the terminal durability barrier rolls
redb back to the prior root. This is proven non-vacuously by the crash-recovery
harness (a torn cross-tree flush at the property↔value-index seam recovers
atomically). One window remains: the embedded
Dataset::querycapture path resolves ops after the in-memory mutation, so a very large single-query write is atomic only up to the flush buffer; the Raft leader closes this by resolving-then-applying against a mirror.
The server
- Native TLS (
--tls-cert/--tls-key) covers REST + management + peer RPCs and the Bolt endpoint (bolt+s:///wss://) (Authentication & TLS). It’s opt-in; without it everything is plaintext (terminate at a reverse proxy instead). - RBAC: global + per-database roles + per-label grants + optional
closed-mode tenant isolation; enforcement is reject, not row-filtering.
Per-user accounts with
admin/writer/readerroles, per-database overrides (GRANT … ON DATABASE …), and per-labelREAD/WRITEgrants (GRANT READ|WRITE ON <db>:<Label> …) exist (Authentication & TLS). By default (open mode) a user’s global role reaches every database, including one it holds no grant on;--rbac-closedopts into real database-level deny for non-admins (see Authentication & TLS). Per-label enforcement rejects a query touching an ungranted (or unclassifiable) label — including one reached only through aWHERE EXISTS {…}, a RETURN pattern comprehension, or aCOUNT {…}subquery — rather than transparently filtering out the ungranted rows; row-level filtering, per-property privileges, and relationship-type/property RBAC (a label-granted user still reads all edge data) are not yet implemented. Auth is fail-closed: an empty password refuses to start unless you pass--insecure-disable-auth. - Security audit log is node-local + in-memory. Authenticated write/admin/DDL
operations and authorization denials are recorded (
GET /mgmt/audit, plustracingtargetpatinadb::audit), but the ring is bounded, not Raft-replicated, and not persisted across a restart; reads are not audited. - No encryption-at-rest. The on-disk graph (and the audit ring) are not encrypted by patinaDB — use OS-level disk encryption. This is a storage-backend change tracked as a follow-on.
- No online membership reconfiguration UI — cluster changes go through the
/mgmt/*HTTP endpoints by hand. - Writes are linearizable through Raft, so write latency includes a consensus
round trip on a multi-node cluster. (A single
--bootstrapnode has no such cost.) - Reads are served from a node’s local applied state by default. On a
follower/learner this can momentarily lag the leader. For a read that reflects
all committed writes, request
"consistency": "linearizable"on/cypher(leader-only, one round-trip — see High Availability). This knob is REST-only; the Bolt path always reads locally. - Explicit Bolt transactions are snapshot-isolated, not serializable.
BEGINpins a consistent snapshot (repeatable reads);COMMITis first-committer-wins (a write-write conflict against a commit made since the snapshot is rejected with a transient, driver-retryable error), which prevents lost updates. Snapshot isolation does not prevent write skew (a read-write conflict with disjoint write-sets), and it is not serializable and not linearizable. For an invariant spanning rows one transaction reads and another writes, use a single autocommit statement or an application-level guard. See Bolt & Neo4j Browser.
Time travel
- Reads only. Time travel never writes. To bring a past state back to the
live graph, use
CALL patinadb.restore('<id>')— it promotes that state to HEAD as a new engram (append-only; see Engrams).
Compatibility
- patinaDB implements a subset of Neo4j. The Bolt endpoint and the system shim are sufficient for the official drivers and the Neo4j Browser, but Neo4j-specific admin surfaces, APOC, GDS, multi-database access control, and enterprise features are not present.
Tracked work
The limitations above that are called out as “not yet” or “a follow-on” are on the roadmap (relationship key/uniqueness constraints, the O(N) sort fast-path integration, lowering list-predicate/subquery forms into the operator tree, edge-sorted-index defs in Raft snapshots, and the deferred full-text items above). If a limitation here blocks you, reach out to your vendor or account representative to check on its status.
Glossary
AttributeValue — patinaDB’s tagged property value type: String, Integer, Float, Boolean, Null, List, Map, the temporal types, and Path.
ANN (approximate nearest neighbor) — a vector search that trades exact recall for sublinear speed. patinaDB’s vector index is ANN via IVF-Flat.
BM25 — the ranking function used by full-text search (k1 = 1.2,
b = 0.75).
Bolt — Neo4j’s binary client protocol. patinaDB’s server speaks it over raw TCP and WebSocket, so Neo4j drivers and the Browser connect directly.
Engram — one committed unit of change: a list of deterministic delta operations plus metadata (id, timestamp, message). The basis of history, diffs, time travel, and replication.
Compound index — a multi-field B-tree index accelerating equality-prefix + sort queries. Maintained automatically.
Dataset — the embedded API: a versioned graph at a directory, with
open / begin / query / commit and the history operations.
DeltaOp — a single low-level mutation (create/delete vertex, set property, set label, create/delete edge). Engrams are lists of these.
Diff (single) — a git-show-style view of one engram.
Diff (range) — a structural, move-aware comparison between two reconstructed graph states.
Fork — create a new database seeded with another’s state at a chosen
engram (FORK DATABASE … [AS OF …] INTO …); the fork gets its own independent
history.
Embedding — a fixed-length list of floats representing an item in a vector space, stored as an ordinary list-valued property. Queried by similarity via a vector index.
Full-text index — a user-defined BM25 inverted index over string properties of a label or relationship type.
IVF-Flat — the vector index strategy: k-means centroids partition the
space into nlist cells; a query scans only the nprobe nearest cells. The
centroids are trained once and replicated, so every cluster node returns
identical results.
Learner — a Raft node that replicates and applies the log but does not vote; acts as an asynchronous read replica.
openraft — the Rust Raft implementation the server is built on (0.9).
Pin — mark a engram so squash never coalesces it, keeping the point-in-time it marks reachable. Tags pin automatically.
Property-value index — a label-scoped, order-preserving index over a
property, enabling O(limit) sorted pagination and efficient equality/range
scans.
Quorum — the majority of voters required for Raft to commit. A 1-node cluster has a quorum of 1 (no redundancy); a 3-node cluster tolerates 1 failure.
Reconstruct — rebuild a past graph state from the nearest snapshot plus forward delta replay; the mechanism behind time travel.
Snapshot — a full-graph capture, taken periodically (every 50 commits) to bound reconstruction cost, and shipped in Raft snapshots to bootstrap a node.
Squash — compact a run of old engrams into one synthetic genesis, keeping recent (and pinned) history; the live graph is unchanged, only the log is compacted.
Tag — a named, pinned, snapshotted reference to a engram (like a git tag), so you can time-travel to a meaningful point via AS OF TAG. Replicates across a cluster.
TCK — the openCypher Technology Compatibility Kit, the conformance suite used to measure Cypher coverage (~95% passing).
Time travel — running a read query against the graph as it was at a past
engram (--at, USE … AS OF, or the at request field).
Vertex / Edge — a node / a directed, typed relationship in the property graph.
Vector index — a user-defined ANN (IVF-Flat) index over an embedding
property, queried with Neo4j syntax (db.index.vector.queryNodes) by cosine or
euclidean similarity.
Voter — a Raft node that participates in elections and commit quorums.