Change Streams (CDC)

patinaDB can stream every committed graph change as it happens, so external systems can react to writes — invalidate a cache, sync a search index, feed a downstream ETL pipeline. This is Change Data Capture (CDC), and it is built directly on the engram log: every commit is an engram of resolved delta-ops, and the change stream is simply that log made subscribe-able.

CDC is a server (Raft) feature, exposed over HTTP as Server-Sent Events (SSE).

The endpoint

GET /changes?db=<name>&since=<engram-uuid>

• db — the database to observe (default default).
• since — a resume cursor: the engram id of the last change you already processed. The stream first replays every engram committed after that id, then live-tails new commits. Omit it (or pass 0) to start from the beginning of history.

The response is an SSE stream (Content-Type: text/event-stream). Each committed write arrives as one change event:

id: 6f9c…-a1b2            ← the engram id = your next resume cursor
event: change
data: {"engram_id":"6f9c…-a1b2","parent_id":"…","db":"default",
       "timestamp":1751900000,"author":"alice",
       "changes":[{"op":"createNode","id":"…","label":"Person"},
                  {"op":"setNodeProperty","id":"…","key":"age","value":30}]}

The SSE id: field is the engram id, so a standards-compliant SSE client resumes automatically via Last-Event-ID after a dropped connection; you can also pass it back explicitly as ?since=.

What timestamp guarantees. It is unix seconds, stamped by the node that proposed the write, and it is non-decreasing along a database’s history: an engram’s timestamp is never older than its parent’s (issue #1077). That holds across a leader failover even when the new leader’s clock is behind the old one’s, so ETL that sorts or windows by timestamp will not see time jump backwards mid-stream.

Two things it is deliberately not. It is not a unique key — several writes within the same second share a second, and a clamped write repeats its parent’s value — so order events by their position in the stream (or by the engram parent chain), never by timestamp alone. And it is not a precise clock: it reflects the proposing node’s wall clock, so it can sit ahead of or behind your own by whatever your fleet’s clock skew is. For exactly-once resumption use the engram id cursor, which is what it is for.

Change records

Each event’s changes array holds one compact record per delta-op:

Resume & delivery semantics

Delivery is at-least-once with the engram cursor. On reconnect with since=<cursor>, the stream replays exactly the engrams after that cursor and then continues live. The server subscribes to the live feed before reading history, so no commit can slip through the gap between “read the past” and “start tailing”; the small replay/live overlap is de-duplicated internally, so a well-behaved consumer sees every engram after its cursor, with no gap and only a bounded, self-healing overlap.

Persist the last engram_id you successfully processed. If your consumer restarts, reconnect with it as since= and you continue exactly where you left off.

Lag

The stream is backed by a bounded in-memory buffer per database. A consumer that falls too far behind receives a lagged event:

event: lagged
data: {"resumeFrom":"6f9c…-a1b2"}

When this happens the server automatically re-reads the engram log from your last cursor (so no changes are lost) and continues. The lagged event is only a cue that a resync occurred.

Snapshot installs (resync from scratch)

When a Raft follower falls too far behind, it catches up by installing a snapshot — a wholesale replacement of the graph and the engram log the change stream reads. A replay that crossed that window would mix the old and new timelines against your old cursor, so /changes fences it the same way ordinary reads do:

• New subscriptions during the install window are refused with a retryable 503 snapshot install in progress; retry — back off briefly and reconnect.

• An established stream that crosses an install (mid-replay or while live-tailing) ends with a terminal error event:

  event: error
  data: {"reason":"snapshot_install","resumeFrom":null}
  

  resumeFrom: null means your old cursor named a discarded timeline and is void. Discard any state you built from this stream, reconnect, and replay from scratch (?since=0, or omit since) — the post-install replay delivers the complete new timeline in causal order. (Do not resume from your last pre-install engram id after seeing this event; for an ordinary error/lagged resync the cursor is still valid, but an install replaces the log it pointed into.)

Retention squash (cursor compacted — resync from scratch)

History retention can squash the oldest engrams into a single genesis engram that carries the compacted state, deleting the collapsed prefix. If your resume cursor sits inside that deleted prefix, the server cannot replay the window between your cursor and the squash cutoff — those changes (including deletes) are gone from the log. Rather than silently replaying the squash genesis’s create-everything event on top of your existing state (a permanent, silent divergence of a downstream index or ETL pipeline), the stream ends with a terminal error event:

event: error
data: {"reason":"cursor_compacted","resumeFrom":null}

The same fence fires mid-replay if a squash lands between the server’s history walk and the moment it reads an engram body it still owed you. A sibling reason, cursor_unknown, means the cursor is not on this database’s chain at all (an id from another database, a typo, or a discarded timeline) — it is handled identically:

event: error
data: {"reason":"cursor_unknown","resumeFrom":null}

Both mean exactly what resumeFrom: null says: your cursor is void. Discard the state you built from this stream, reconnect, and replay from scratch (?since=0) — a from-scratch replay always works, because the squash genesis itself carries the complete compacted state. To avoid this, keep your consumer’s lag comfortably inside the retention window (squash keeps a configurable number of recent engrams), or pin the engram your pipeline is anchored to.

A caught-up consumer is never fenced: subscribing with since= equal to the current HEAD (or omitting since) is an empty replay followed by the live tail, exactly as before.

Consistency (read this)

The change stream is node-local and eventually consistent, exactly like a consistency=local read:

• It observes this node’s applied HEAD as it advances. On a follower it lags the leader slightly; it never reflects an uncommitted write.
• Ordering within a database is the engram chain order (each event’s parent_id is the previous event’s engram_id).
• Publishing a change never blocks replication — a slow or dead subscriber can never stall the write path; it just lags and resyncs.

For a single-writer-of-record consumer, subscribe to the leader (see cluster routing); a follower stream is fine for best-effort reactions where a small delay is acceptable.

Authorization

/changes is authorized as a read on the target database — the caller needs at least the Reader role for that db (see Authentication & RBAC). An unauthenticated request is rejected with 401; in closed-mode tenant isolation, an ungranted database is 403.

A fine-grained (per-label or strict-relationship) user cannot subscribe at all. A ChangeRecord streams raw property values and full label sets with no per-event label filtering, so the row-level filtering that protects an ordinary /cypher read (see per-label grants) can’t be applied to a live firehose — a subscriber holding any per-label grant, or any relationship grant under strict-rel mode, is refused the subscription outright (403) rather than risk it observing an ungranted label’s data. A user with only a blanket database role (no fine-grained grants) is unaffected and streams normally.

Example

# Tail the default database's changes (with credentials + resume cursor).
curl -N -u alice:apw \
  'http://localhost:8080/changes?db=default&since=6f9c…-a1b2'

-N disables curl’s buffering so events print as they arrive. Any SSE-capable client (browser EventSource, an SSE library in your language) works the same way.

Related: CDC is a live view over the same engram log that powers diffs and time travel. Where those read history on demand, CDC pushes each new engram as it commits.