Bulk Loading & Import

patinaDB offers three ways to get a lot of data into a graph, from fastest and least featureful to fully online and versioned:

Offline: patinadb import

The offline bulk-loader reads neo4j-admin-style CSV, Parquet, or Arrow IPC files and writes them straight into the embedded store in durable batches, bypassing Raft and the engram/versioning layer. That is what makes it fast — and it means the loaded data is visible at HEAD but is not in --at time-travel history. All node files load first, then all relationship files.

patinadb ./mygraph.db import --nodes people.csv --nodes companies.csv \
                             --rels works_at.csv

Real run (three node rows + two company rows + three edges):

imported 5 nodes (11 props), 3 edges (3 props) in 0.00s (1058 nodes/sec)

File format

Files follow the neo4j-admin convention. Node files carry a :ID column, a :LABEL column, and typed property columns; relationship files carry :START_ID / :END_ID / :TYPE plus property columns.

:ID,name,age:int,active:boolean,:LABEL
p1,Ada,36,true,Person
p2,Grace,40,true,Person

:START_ID,:END_ID,:TYPE,since:int
p1,c1,WORKS_AT,2018

• CSV property types come from a name:type header suffix — :int, :float, :boolean, :date, :localdatetime, or a bare name for string.
• Parquet / Arrow property types come from the file’s Arrow schema (Int*/UInt* → Integer, Float* → Float, Utf8 → String, Boolean → Bool, Date32/64 + Timestamp → temporal); an unsupported column type is a loud error, not a silent drop. A null cell means the property is absent.
• Dispatch is by extension (.csv / .parquet / .arrow / .feather / .ipc) or forced with --format.

Deterministic ids and idempotent re-runs

By default (--id-type hash) each :ID string is hashed to a deterministic UUIDv5. This is what lets a relationship file reference endpoints by their business key with no id-map, and it makes a re-run idempotent (the same :ID always maps to the same vertex UUID).

With --id-type uuid (or an auto-detected :ID(uuid) header) the id cell is the literal vertex UUID. This is the round-trip mode: an export writes :ID(uuid), so re-importing reproduces the same graph, same UUIDs included.

patinadb ./mygraph.db export --out ./dump --format parquet
patinadb ./fresh.db import --nodes ./dump/nodes_Person.parquet \
                           --rels  ./dump/rels_KNOWS.parquet

See the CLI reference for every flag (--batch-size, --delimiter, --skip-ref-check, …) and the matching export command. On the server, GET /mgmt/export streams the same import-compatible files as a tar archive (see the REST API).

Throughput reality

Bulk load is value-index-write-bound, not parse-bound: every property write also maintains the label-scoped value index. In practice this is roughly 25–38k nodes/sec on one box, and Parquet is not meaningfully faster than CSV — the columnar reader is quicker, but the index writes dominate. Memory stays O(batch-size) regardless of file size (streaming).

Online: LOAD CSV

LOAD CSV streams rows from a CSV file into a running query as a row source — like UNWIND, but from a file. Unlike the offline importer, it goes through the normal write path, so each load is versioned (an engram) and, on a cluster, replicated.

LOAD CSV WITH HEADERS FROM 'file:///data/people.csv' AS row
CREATE (:Person {id: toInteger(row.id), name: row.name})

• WITH HEADERS makes each row a map keyed by header (row.name); without it each row is a list of string cells (row[0]).
• Cells are strings — coerce with toInteger / toFloat / toBoolean. A coercion may sit directly in a CREATE/MERGE pattern property (as above); a row[i] list-index expression still needs a WITH stage first.
• FIELDTERMINATOR '<c>' overrides the , delimiter. http(s):// URLs are rejected; only file:// and bare/relative paths are read.

The full clause reference is in Cypher Support → Loading CSV.

The un-chunked memory cliff

A bare LOAD CSV … CREATE streams the source row by row, but buffers all resolved write operations into one transaction (on the server, one Raft entry) before committing — an O(rows) memory cliff for a large load. A LOAD CSV … RETURN row read query streams end to end and has no such cliff.

Online + chunked: CALL { … } IN TRANSACTIONS

Wrapping the write subquery in CALL { … } IN TRANSACTIONS chunks the load into many small, independently-committed transactions — the online answer to the RAM cliff. Peak memory drops to one chunk, and on a cluster each chunk replicates as one small Raft entry instead of one unbounded one.

LOAD CSV WITH HEADERS FROM 'file:///data/more_people.csv' AS row
CALL {
  WITH row
  CREATE (:Reader {id: row.id, name: row.name, age: toInteger(row.age)})
} IN TRANSACTIONS OF 2 ROWS

Loading a 5-row file OF 2 ROWS produces ⌈5/2⌉ = 3 chunks — and three separate engrams, one per committed chunk:

$ patinadb ./mygraph.db query "MATCH (r:Reader) RETURN count(r) AS n"
n: 5

$ patinadb ./mygraph.db log | wc -l
3

• Each chunk is its own commit ⇒ its own engram, so batched ingest is versioned and time-travellable (unlike the offline importer). A later chunk sees the writes committed by earlier chunks (read-your-writes).
• OF <n> ROW[S] sets the chunk size (default 1000).
• ON ERROR — FAIL (default) aborts the whole statement on a failing chunk; CONTINUE skips it and commits the rest; BREAK stops after it, keeping earlier chunks. A skipped/failed chunk changes nothing — it is resolved against a throwaway copy of the graph, so a partial chunk is never left behind.
• Crash-atomic per chunk: a crash between chunks recovers to a chunk boundary. The graph and the engram log always agree.
• The CALL { … } IN TRANSACTIONS must be the final clause of the query, and it cannot run inside an explicit Bolt BEGIN … COMMIT transaction.

On the server the driver proposes one client_write per chunk, so a 250-row load OF 100 ROWS advances the applied index by 3, not 1 — bounded Raft entries, read-your-writes preserved across chunks on the leader.

Server file-I/O sandbox

Reading a file:// URL, or writing with the export procedures, touches the server’s filesystem. On the server both are deny-by-default and gated by two layers:

1. Directory sandbox — --allow-csv-dir <dir> (reads) and --allow-export-dir <dir> (writes) whitelist directories; unset means every file access is refused. Paths are canonicalized, so .. traversal and symlink escapes are rejected.
2. Authorize by effect — any query that does file I/O is raised to require the global Admin role, because file access is a host-level capability, not a graph-data one. A per-database Writer cannot read or write host files.

The embedded library and the CLI install no sandbox, so they are unaffected (local file access is the point). See Authentication & TLS → Cypher-driven file I/O.

Which path should I use?

• A one-off migration into a fresh database, or millions of rows → offline patinadb import (fastest; give up in-history versioning of the load).
• Live ingest into a running / clustered database → LOAD CSV … CALL { … } IN TRANSACTIONS OF n ROWS (versioned, replicated, bounded memory).
• A small load, or you need the CSV as a read source → plain LOAD CSV … CREATE / LOAD CSV … RETURN row.