Anamnesis

Anamnesis is patinaDB’s provenance/lineage system.

patinaDB is versioned by construction — the engram log already records what changed and when. Anamnesis turns that stream into a queryable W3C PROV-style property graph: for every write, who did it (an agent), the commit that did it (an activity), and — at the label / type and property level — what kind of change it made and how much.

The projection lives in a separate companion database named <db>__anamnesis, isolated from your main graph so it never touches your graph’s trees or indexes (main-graph read performance is unaffected). Because it is an ordinary database, it is replicated and carried in Raft snapshots for free, and you query it with plain Cypher.

The model

Provenance is aggregated by label/type + property, not one node per touched vertex. A write that creates a million :Ticket rows projects a handful of provenance nodes (one per label and property touched), not a million — the concrete uuids are not duplicated into the PROV graph (see Drilling to concrete uuids).

In <db>__anamnesis, each write projects to:

where op ∈ {created, updated, deleted} and on ∈ {node, edge}, and the relationships:

• (:Activity)-[:WAS_ASSOCIATED_WITH]->(:Agent)
• (:Activity)-[:AFFECTED {count, op}]->(:NodeType | :EdgeType)
• (:Activity)-[:SET {count}]->(:Property)

Agent, NodeType, EdgeType and Property are upserted by a content-derived stable id, so repeated writes never duplicate them: a (:NodeType {label:'Ticket', op:'created'}) touchpoint is created once and reused by every activity that creates Tickets. Each commit produces exactly one new :Activity, with AFFECTED/SET edges from that activity carrying the per-label counts.

Why op is on the type node. patinaDB edges are keyed by their (outbound, type, inbound) triple with no independent edge id, so two AFFECTED edges from the same Activity to a shared per-label node would collide. The created / updated / deleted touchpoints are therefore distinct :NodeType nodes (one per (label, op)), and MATCH (:NodeType {label:'X'}) returns up to three of them.

How ops map to touchpoints:

• AFFECTED (nodes): a CREATE → (label, created); a DELETE → (label, deleted); a property/label set or removal on a pre-existing node → (label, updated). Property/label sets on a node created in the same commit fold into created (creating a node with properties is one act, not a create-then-update).
• AFFECTED (edges): the same, keyed by the relationship type.
• SET (properties): every value set — SetVertexProperty / SetEdgeProperty, including on freshly created entities — records a (label, key) :Property touch. This is the property-level tracking axis: “which activity last set Ticket.status?”

The agent is the authenticated RBAC user, or "anonymous" when authentication is disabled. The timestamp is the leader-stamped commit time carried in the Raft entry, so it is identical on every replica.

Opt in

Provenance is off by default (zero cost when off — the write path does no provenance work). Enable it per database:

CALL patinadb.anamnesis.enable()      -- turn it on for the current/USE'd db
CALL patinadb.anamnesis.disable()     -- turn it off (companion data is kept)

USE sales CALL patinadb.anamnesis.enable() targets sales. Enabling creates the sales__anamnesis companion (idempotent) and replicates to every node. Requires the global Admin role.

Querying

The companion is a normal database — the primary path is a plain USE. Query provenance edges in the outbound Activity → target direction:

USE mydb__anamnesis
MATCH (a:Activity)-[r:AFFECTED {op: 'created'}]->(nt:NodeType {label: 'Ticket'})
RETURN a.timestamp AS when, r.count AS how_many
ORDER BY a.timestamp DESC

“Which activities set Ticket.status, and how often?”

USE mydb__anamnesis
MATCH (a:Activity)-[s:SET]->(:Property {label: 'Ticket', key: 'status'}),
      (a)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
RETURN ag.name AS who, a.timestamp AS when, s.count AS n
ORDER BY a.timestamp DESC

Query direction matters. Reading an edge property (r.count, r.op) across an inbound <- traversal currently returns NULL — a general engine limitation, not specific to provenance. Always traverse out of the :Activity ((a)-[r:AFFECTED]->…) when you need the edge’s count/op.

A convenience read procedure returns the activities that touched a node-label (newest-first):

CALL patinadb.anamnesis('Ticket')
YIELD engram_id, timestamp, message, op, count, agent

(It is server-side sugar for the USE <db>__anamnesis MATCH … query above. The argument is a label, not a vertex uuid — provenance is label-aggregated.)

The main-database engram log also records the writer: CALL patinadb.engrams() YIELD id, author, timestamp shows who committed each engram.

Drilling to concrete uuids

The PROV graph is deliberately coarse: it tells you which labels and properties an activity touched and how many — not which uuids. To get the concrete vertices a commit changed, drill into the engram delta log with the Activity’s engram_id:

CALL patinadb.diff('<engram_id>')   -- the per-vertex added/removed/changed detail

So per-vertex blame (“who last wrote this node”) is a diff-scan over history (cost O(history)), not an O(1) lookup. A reverse uuid → activities index is a possible future add if that access pattern becomes hot.

Enrichment

Auto-projected provenance records the structure of a write (who / what-kind / how-many). Enrichment lets the client attach context to the write — the run/source/model of the activity — so a pipeline can say why and from where it wrote, not just that it wrote. This is activity-level enrichment: each context entry becomes a property on that commit’s :Activity node.

It costs nothing unless you use it, and it is ignored (no error) when provenance is disabled for the target database.

Attaching a context

REST /cypher — add an optional provenance object (an arbitrary key→scalar map) alongside the query:

{
  "query": "CREATE (:Doc {path: 'src/main.rs'})",
  "provenance": {
    "agent":  "indexer-run-42",
    "source": "git://repo@abcd123",
    "model":  "text-embedding-3-large"
  }
}

Bolt — drivers send it as transaction metadata, the canonical Neo4j mechanism. patinaDB reads tx_metadata from the RUN extra (autocommit) and the BEGIN extra (explicit transactions):

# autocommit
session.run("CREATE (:Doc {path: $p})", p="src/main.rs",
            metadata={"agent": "indexer-run-42",
                      "source": "git://repo@abcd123",
                      "model": "text-embedding-3-large"})

# explicit transaction — metadata set once at begin, applies to the whole tx
tx = session.begin_transaction(metadata={"agent": "indexer-run-42",
                                         "source": "git://repo@abcd123"})
tx.run("CREATE (:Doc {path: $p})", p="a")
tx.run("CREATE (:Doc {path: $p})", p="b")
tx.commit()   # one Activity, enriched with the begin metadata

Reserved keys

Some keys are special and are consumed (never folded onto the :Activity):

• agent (or its alias actor) sets the :Agent name instead of becoming an Activity property — so the recorded agent can be a pipeline or tool (e.g. "indexer-run-42", "nightly-etl") rather than the authenticated RBAC user. When both agent and actor are given, agent wins. Without either, the agent stays the authenticated user (or "anonymous").
• confidence, source and derived_from are the provenance values. They attach at a client-chosen granularity — see Scope: where the values attach below. By default (scope: changeset) they become :Activity properties, which is cheap.
• scope selects that granularity (changeset | label | attribute | instance). It is consumed, never emitted.

Every other key becomes an ordinary :Activity property.

Querying enrichment

Enrichment props are ordinary :Activity properties, and the overriding agent is an ordinary :Agent:

USE mydb__anamnesis
MATCH (a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
WHERE a.model = 'text-embedding-3-large'
RETURN ag.name AS agent, a.source AS source, a.timestamp AS when
ORDER BY a.timestamp DESC

Rules and limits

• Determinism. The context is folded into the projection on the Raft leader and baked into the prov ops carried in the same Raft entry, so every node records a byte-identical enriched Activity.
• Coercion (best-effort — enrichment never fails the main write). Values are coerced to a stored scalar: strings/numbers/booleans are kept; null values are dropped; nested lists/maps are stringified to JSON. Strings are truncated to 4096 characters, and at most 32 keys are folded (excess dropped in sorted order). Malformed or oversize metadata is clamped, never rejected.
• Structural keys are protected. engram_id, timestamp and message are owned by the projector; a context key with one of those names is ignored (it cannot overwrite the built-in Activity fields).

Scope: where the values attach

The three provenance values (source / confidence / derived_from) attach at a granularity you choose with the reserved scope key. One write supplies one set of values, applied uniformly to every target at that scope. This lets you record provenance cheaply at the commit level by default, and pay the per-entity cost only when you ask for it.

scope is case-insensitive; activity is an alias for changeset; an unknown value falls back to changeset (with a warning). If none of source / confidence / derived_from is supplied, scope is irrelevant and nothing extra is emitted — an ordinary write pays only for the label/type touchpoints.

Coercion is best-effort and never fails the write: confidence → float, source / derived_from → string.

source is cheap by default. Because the default scope is changeset, a bare source (or confidence) lands on the single :Activity node — it does not create per-entity nodes. Per-instance cost is paid only when you explicitly ask for scope: instance.

changeset (default) — commit-level

{
  "query": "CREATE (:Ticket {status: 'open'})",
  "provenance": { "source": "git://repo@abcd", "confidence": 0.9 }
}

source and confidence become :Activity properties (scope omitted ⇒ changeset):

USE mydb__anamnesis
MATCH (a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
WHERE a.source = 'git://repo@abcd'
RETURN ag.name AS who, a.confidence AS confidence, a.timestamp AS when

label — per touched node/edge type

{
  "query": "MATCH (t:Ticket) SET t.status = 'closed'",
  "provenance": { "scope": "label", "source": "bulk-migration-7" }
}

The value rides every AFFECTED edge of the write:

USE mydb__anamnesis
MATCH (a:Activity)-[r:AFFECTED]->(nt:NodeType {label: 'Ticket'})
RETURN nt.op AS op, r.count AS n, r.source AS source

attribute — per touched (label, key)

{
  "query": "MATCH (t:Ticket) SET t.priority = 3",
  "provenance": { "scope": "attribute", "source": "triage-rules-v2", "confidence": 0.8 }
}

The value rides every SET edge:

USE mydb__anamnesis
MATCH (a:Activity)-[s:SET]->(p:Property {label: 'Ticket', key: 'priority'})
RETURN s.count AS n, s.source AS source, s.confidence AS confidence

instance — per changed entity (the expensive opt-in)

This is the shape an extraction / LLM pipeline needs when it asserts a fact per write: “for this specific entity, which run generated it, with what confidence, from what source?”

{
  "query": "CREATE (t:Ticket {summary: 'db is slow'})",
  "provenance": { "scope": "instance", "agent": "run-42", "source": "doc://y", "confidence": 0.9 }
}

For each vertex created or updated in the write, in <db>__anamnesis:

• (:Entity { ref_uuid, ref_label }) — one per concrete main-graph entity, keyed by a uuid5 of its ref_uuid (idempotent upsert: repeated writes to the same entity reuse the node). ref_label is the entity’s primary label.
• (:Activity)-[:GENERATED { confidence?, source?, derived_from? }]->(:Entity) — one per (activity, entity), carrying the reserved values as edge properties.

The reused :Agent / :Activity are the same Layer-1 nodes (instance scope never duplicates them). Deleted vertices are skipped — per-instance provenance of a now-deleted entity is out of scope.

Per-entity provenance — “which run generated this ticket, with what confidence?” (read the edge props traversing out of the :Activity, or across an inbound <- — the latter is supported):

USE mydb__anamnesis
MATCH (e:Entity {ref_uuid: '…'})<-[g:GENERATED]-(a:Activity)-[:WAS_ASSOCIATED_WITH]->(ag:Agent)
RETURN ag.name AS run, g.confidence AS confidence, g.source AS source

All low-confidence entities:

USE mydb__anamnesis
MATCH (a:Activity)-[g:GENERATED]->(e:Entity)
WHERE g.confidence < 0.5
RETURN e.ref_label AS label, e.ref_uuid AS uuid, g.confidence AS confidence
ORDER BY confidence ASC

Determinism

At every scope the values are folded into the projection once on the Raft leader and baked into the prov ops carried in the same Raft entry, so the result — Activity props, edge decorations, or the per-instance graph — is byte-identical on every node.

One source per scope, per write (future: per-target)

A write supplies one set of values, applied uniformly to all targets at the chosen scope. Per-target distinct sources — e.g. a different confidence for each changed property under scope: attribute, or per-entity under scope: instance, expressed as a nested map — is a possible future extension and is not built. Likewise, field-level provenance is the finest attribute scope offers today (per (label, key), not per (entity, key)).

Honest caveats

• Write amplification is now O(labels + properties), not O(nodes). A write projects a handful of touchpoint upserts + per-label count edges regardless of how many rows it touched. It is still non-zero work per write — keep provenance off during bulk loads if you don’t need it, and enable it afterward (the companion only reflects writes made while it was enabled).
• Per-uuid blame is a diff-scan, not a graph lookup — see Drilling to concrete uuids.
• Server path only. Provenance is projected on the Raft server write path (REST /cypher and Bolt). Embedded Dataset::query writes are a follow-up (the projector lives in the core library and is reusable via Dataset::build_provenance_ops).
• Unresolvable labels bucket under "?". A vertex property/delete op carries only a uuid; the leader resolves its label from the live graph. In the should-never-happen case that a label can’t be resolved, the touch is bucketed under label "?" (and logged) rather than dropped.
• Activity ↔ engram correlation is best-effort. Activity.engram_id is the main engram id predicted at propose time; under concurrent-write interleaving it could differ from the recorded id. The provenance graph itself is always internally consistent and byte-identical across replicas.