Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Query Planning & Performance

patinaDB compiles each query into a streaming operator tree (the “Op tree”), picks a physical access path for every entry point, and — on skewed data — uses a statistics catalog and a cost model to choose between competing plans. This chapter shows how to read what the planner chose and which fast paths make a query cheap.

Start with How the Query Planner Works. That chapter is the structured reference: the planning pipeline, entry and join choice, statistics, the experimental RFC 0042 switches and how to read the Decisions: section. This chapter keeps the detailed per-feature notes.

Immediate IN operands

When an IN predicate has a single parameter or literal as its right-hand operand, execution borrows that value for the membership check, including inside compound Boolean predicates. The source value remains owned by the statement or plan. This avoids making a new owned RHS copy for every candidate row. Row-dependent RHS expressions continue through the ordinary evaluator. Query memory limits still apply to owned inputs, intermediate work and results; this behavior introduces no configuration switch or higher limit.

WHERE placement

A WHERE comparison of a property with a literal or parameter is checked inside the pattern itself. This applies in either operand order ($d > m.date is the same as m.date < $d), in chained comparisons, and in every MATCH after a WITH. It applies only to the MATCH clauses written after the last UNWIND, CALL or LOAD CSV before the WHERE, so a filter never skips an error raised by such a clause.

WHERE n:Label on a node of a required pattern becomes a pattern label, so the label can choose the scan. A comparison of a single relationship’s property with a literal or parameter (WHERE r.since > $d) is checked while the relationship is expanded; a failure to read that property fails the query rather than dropping the row.

In a MATCH with several comma-separated patterns, a condition that reads only one of the joined patterns runs on that pattern before the next one is expanded. This covers =, <> and IS [NOT] NULL on its variables, properties, literals and parameters, combined with NOT, AND and OR. Conditions that can raise an error stay where they were written.

Reading EXPLAIN and PROFILE

For a direct property projection, RETURN n.key AS key ORDER BY key can use the same ordered property scan as ORDER BY n.key for an eligible single-node shape, subject to index/population safety checks. Traversals, DISTINCT and multi-key ordering keep their previous scan hints and post-sort behavior. Computed aliases such as toLower(n.key) do not imply that the original property’s index order is usable. An ascending STARTS WITH scan can seek to the prefix lower bound while retaining residual filters before applying the output window. Deep SKIP still requires advancing past qualifying rows; use keyset pagination when the application can express a stable cursor. These are generic planner rules, not special cases for ticket IDs.

EXPLAIN renders the chosen plan without running the query; PROFILE runs it and prefixes the elapsed time. For buffered Bolt EXPLAIN/PROFILE requests outside an explicit transaction, the effective client/server query deadline covers the complete execution, including time waiting for a blocking worker. Native execution runs off the async connection thread. Deadline enforcement remains cooperative; it cannot interrupt every individual storage call. No separate profiling timeout is used.

EXPLAIN MATCH (p:Person {name: 'Ada'}) RETURN p
plan: 'Project items=1 distinct=false return_star=false
  MatchScan required=[[entry](p:Person) → PropertyIndex[name] = est.1]
'

Read the tree bottom-up: the MatchScan is the leaf that produces rows, and each line above consumes them. The arrow annotation is the access path and its estimated cardinality — here a PropertyIndex[name] = est.1 point lookup on the value index (one estimated row), not a full label scan.

A range predicate with an ORDER BY that a value index can serve turns into a sorted seek and emits a Physical: footer naming the executed access path:

EXPLAIN MATCH (r:Reader) WHERE r.age > 40 RETURN r.name ORDER BY r.age
plan: 'Project items=1 distinct=false return_star=false
  MatchScan required=[[entry](r:Reader) → PropertyIndex[age] SORTED SCAN ASC est.2]
Physical:
  entry.range_seek
'

The Physical: footer describes the planned physical rules from the shared registry. Runtime admission and preparation can still retain another path; use PROFILE’s executed-access observations to distinguish the actual choice from the plan. Common footer names:

FooterMeaning
entry.range_seekValue-index range seek (aligned >/>=/</<= bound).
entry.cost_override: <rule> (≈N rows) beats <priority-rule> on costThe cost model overrode the default priority pick (see below).
limit.bounded_topkORDER BY … LIMIT k via an O(k)-memory top-K heap (no full sort).
limit.edge_sorted_topkTraversal + ORDER BY target.prop LIMIT k served by an edge-sorted index.
limit.reverse_probe (fanout est.N → probe est.M)Same shape as above with no edge-sorted index: instead of expanding the whole adjacency and sorting it, the planner walks the target label’s value index in sort order and probes each candidate for the edge, stopping once LIMIT is filled. Cost-chosen against limit.bounded_topk (the two estimates are shown), and never preferred over a real edge-sorted index.

Choosing an equality property

For an unsorted pattern with several eligible equality properties, the property rule compares bounded index probes instead of using the first property in the query. For example, a shared run_id need not drive every lookup when id identifies a much smaller set. The selected predicate and all other conditions are still enforced, including repeated conditions on the same property.

Each candidate probe shares a raw-key budget across its numeric representations and any overlay source and local entries. Probes that reach the limit tie by field name and encoded value; they do not compute a full distribution. The estimates use committed data and can omit pending buffered writes. This can change which plan is preferred, but execution still reads its own writes. These estimates never prove that a result is absent. Existing UUID, IN-list and sorted access rules retain their precedence. EXPLAIN can separately collect its existing cardinality estimates from the statistics catalog.

For label-only entry selection, a zero count in the primary-label index does not prove that a registered label is empty: vertices may carry it as an additional label. In that case the planner uses a coarse source-total estimate, or an unknown estimate when the source total is zero. A label absent from the registry can still estimate zero. This changes plan choice, not matching or result completeness. Positive primary-label estimates are retained and may still underestimate labels that also have secondary holders. There is no new setting for this fallback.

Secondary-label decoding keeps up to 16 successful registry ID-to-label mappings per storage manager in fixed inline slots. A slot collision uses the ordinary registry lookup; missing IDs and errors are not cached. The memo is local to its storage view and remains valid across sync and graph clearing because registry IDs are never recycled. It does not cache a vertex’s current label membership. There is no new configuration setting; query memory limits and error handling are unchanged.

For multi-pattern inner joins, the planner can check an unestimated fixed-length path earlier when at least two distinct nodes in that path are already bound, or when it is a closed path whose starting/closing node is already bound. It uses the existing seeded nested-loop execution instead of building the entire unbound path. The initial driving pattern stays unchanged. Variable-length and open single-anchor paths do not qualify for this rule. This is a heuristic: unknown fan-out can still make a plan expensive, and no latency bound is implied. It adds no setting and does not change matching, edge uniqueness, or query deadlines.

The statistics catalog

The planner keeps a lazy, cached statistics catalog. Per label it holds a vertex count, taken from the primary-label index. Per property it holds column statistics from the value index: the present count, distinct-value count (NDV), min/max and a 16-bucket equi-depth histogram. CALL patinadb.stats computes the same quantities exactly and fresh, including secondary-label holders, so its output can differ from the planner’s cached, primary-label-only estimates:

CALL patinadb.stats('Person')

For a small Person dataset (3 nodes) this yields, per property:

propertycountndvminmax
(label)3———
active32falsetrue
age333240
name33AdaGrace

In the procedure output the label summary row has a NULL property. The catalog is computed on demand and cached in RAM only. Each entry is tagged with its label’s write generation and served until it is more than PATINADB_STATS_STALENESS_BUDGET generations (default 64) old. A generation advances once per dirty label per storage synchronisation. Because statistics only change which plan runs, never the result, they are never replicated across a cluster; each node computes its own. The degree catalog and freshness rules are described in How the Query Planner Works.

Cost-based selection

Two things use the catalog:

  1. Entry-point selection. For a skewed EQ predicate (a value with many rows) or a range predicate, the estimator tightens its guess using the catalog (present_count / NDV, or histogram interpolation) instead of a pessimistic whole-label count. That can move the entry point of a multi-node pattern to the genuinely cheapest node.

  2. Physical-rule ranking. When both a covering compound index and a selective single-property seek apply to the same node, the planner ranks them by an estimated Cost { rows, io } rather than static priority — so a non-selective compound index correctly loses to a selective single-property seek. When the cost model overrides the default pick, EXPLAIN’s footer says so (entry.cost_override: …).

  3. Join ordering. A multi-pattern comma-MATCH is reordered so the most selective pattern drives the join and the intermediate result stays small (a bounded Selinger DP for ≤ 8 patterns, greedy above that). Connectivity is preserved, so the reorder never introduces a cartesian product the written order avoided.

On valid data, join order and access-path changes preserve the result multiset. Different access paths can visit different records and change storage-error order, or whether corruption in an unrelated record is observed. Errors from accessed records still propagate. A query without ORDER BY returns rows unordered; one with ORDER BY post-sorts.

Fast paths that make queries cheap

The executor recognizes a number of shapes and serves them without a full scan or a blocking sort. You don’t opt into these — they fire automatically when the shape and the available indexes match. The Physical: footer tells you which fired.

ShapeFast path
MATCH (n:L {p: v})Value-index point lookup (PropertyIndex[p] =).
WHERE n.p IN [...]IN-list seek — a union of point seeks. Eligible secondary-label matches use primary-label value seeks and membership checks; larger label/probe counts retain the secondary scan.
WHERE n.p STARTS WITH 's'Prefix-range scan over the order-preserving string index.
WHERE k1=X AND k2 > Y ORDER BY k2 (compound (k1,k2))Compound range-seek to the range boundary.
ORDER BY p [DESC] LIMIT k (index-served)O(log N + k) sorted scan, LIMIT pushed in.
ORDER BY p [DESC] SKIP n LIMIT kDeep-SKIP key-only cursor advance (no per-skipped-row fetch).
WHERE p > $cursor ORDER BY p LIMIT kKeyset pagination — O(log N + k) value-cursor seek.
ORDER BY a, b [… ASC/DESC mixed] [LIMIT k] (single label, stats-gated)Covering-compound (same direction) or leading-key partial-prefix scan (any direction mix) — no post-sort.
ORDER BY … LIMIT k (not index-served)Bounded top-K heap — O(k) memory, not a full sort.
traversal + ORDER BY target.prop LIMIT kEdge-sorted top-K when a covering index exists.
[NOT] EXISTS { (n)-[:R]->() }Bare pattern-existence probe — one edge-index seek, no sub-plan.
WHERE NOT EXISTS { (n)-[:R]->(m) } (single hop, no inner WHERE)Anti-join memo (exists.anti_join) — the existence verdict is answered once per distinct anchor node, not once per outer row.
RETURN n.p, count(*) grouped by n.pGroup-by-count run-length value-index scan.
MATCH (n:L) RETURN count(n) / count(*) (no WHERE)Label-count key-only label-index count (agg.label_count_scan) — no vertex reads at all. Declines when L is also used as a secondary label.
MATCH ()-[r]->() RETURN count(r) / count(*)Exact global relationship count (agg.global_directed_edge_count): reads current forward-table cardinality for a bare directed one-hop pattern. Filters, labels, grouping, DISTINCT, repeated aliases, overlays and explicit liveness probing retain ordinary execution.
MATCH ()-[r:T]->() RETURN count(r) (also T1|T2)Typed relationship count (agg.typed_directed_edge_count): one key-only pass over the forward edge table comparing the stored type id, without the all-node scan or per-node seeks. Still proportional to the total number of relationships (unlike a maintained count store); the same pattern restrictions as the global count apply.
Fixed linear MATCH with unequal terminal nodes, RETURN count(*)Terminal inequality count (agg.endpoint_inequality_count) for supported paths: streams interior matches and subtracts equal endpoint identities from terminal counts. Requires query memory admission and a proof of relationship uniqueness; unsupported predicates/shapes retain ordinary execution. It may reduce row work without reducing every kind of read; no general latency improvement is guaranteed.
WHERE distance(n.p, c) < D / withinBBox(...)Spatial curve-range seek (entry.spatial_seek) when a point index exists.
ORDER BY distance(n.p, c) LIMIT kSpatial kNN expanding-ring search over a point index.

The advisor

When a traversal + ORDER BY target.prop LIMIT k shape could be served by an edge-sorted index but none exists, both EXPLAIN and the advisor procedure tell you exactly which index to create:

EXPLAIN MATCH (c:Company {name:'Acme'})<-[:WORKS_AT]-(p:Person)
RETURN p ORDER BY p.age DESC LIMIT 3
plan: 'Limit skip=0 count=3
  Project items=1 distinct=false return_star=false
    Sort keys=1
      MatchScan required=[(c:Company) -[entry](p:Person) → PropertyIndex[age] SORTED SCAN DESC est.3]
Physical:
  limit.bounded_topk
Advice: CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age  (would serve this ORDER BY … LIMIT via edge_sorted_topk instead of a fan-out top-K)
'

The same suggestion is available programmatically:

CALL patinadb.advisor(
  'MATCH (c:Company {name:"Acme"})<-[:WORKS_AT]-(p:Person) RETURN p ORDER BY p.age DESC LIMIT 3'
) YIELD suggestion
suggestion: 'CREATE EDGE SORTED INDEX FOR (m:Person)<-[:WORKS_AT]-() ON m.age'

Creating that index flips the plan to limit.edge_sorted_topk and the advice disappears. See Edge-Sorted Indexes.

Bounded-memory guards

Reducible aggregates consume input incrementally; their group state and DISTINCT/collect buffers remain bounded. Blocking operators (non-top-K ORDER BY, UNION-distinct, and hash-join build sides), plus the row expansion of UNWIND inside a correlated subquery, pattern-comprehension result lists, and each partial-prefix sorting group (including the NULL group), are capped at PATINADB_MAX_AGG_ROWS (default 5,000,000). Past the cap they raise a clear, actionable error instead of an OOM — the default is far above any normal query, so below it the result is byte-identical. See Configuration and Caching & Memory Tuning for the RAM budget model, and Cache Observability & Tuning for the read-cache hit rates that make repeated hot queries cheap.

Subquery planning and warnings

Scalar expressions share recursive planning in RETURN, WITH, WHERE and function arguments. This includes COUNT/EXISTS nested in lists, maps, CASE, list comprehensions, REDUCE and ALL/ANY/NONE/SINGLE. Iterator/accumulator variables are local: [x IN [1,2] | x] uses its own x even if an outer x exists. NULL/empty-list quantifier semantics and short-circuiting are preserved.

For eligible single-pattern bodies, COUNT counts matches incrementally and EXISTS stops on a successful match. Complex WITH/RETURN tails can still use bounded materialization; pattern comprehensions collect bounded result lists. Subquery storage errors, query timeouts and memory-limit errors fail the query; they are not zero counts, false existence results or partial lists. Missing properties still yield NULL; failed property reads fail the query, including dynamic/entity-valued access and traversal target-property filters. Unused CASE/coalesce branches do not perform those reads.

EXPLAIN and PROFILE append deduplicated Warning [code] lines:

CodeExecution boundaryPossible change to evaluate
PDB1230-COUNTCOUNT retains per-input execution outside eligible projection reuse and bounded equality grouping; no general decorrelationUse EXISTS/NOT EXISTS if only existence is needed. For repeated exact counts, consider a grouped CALL/join.
PDB1230-EXISTSAn EXISTS body with operators beyond a scan and optional filter materializes its bounded subplan resultFilter outer rows first. Simplify to a pattern only when aggregation, pagination, errors and empty-input behavior are preserved.
PDB1230-COLLECTPattern comprehension materializes a list per input rowConsider COUNT or EXISTS with the same pattern and filter only if discarding projection evaluation is acceptable: projection errors and volatile expressions can change behavior. Use MATCH to return streamed rows if that result shape is suitable.
PDB1230-APPLYIdentifies either bounded reuse eligibility for identical proven CALL dependencies, or complete declared imports when unproved (complete inputs for internal APPLY) or a plan not proven repeatable; distinct inputs still execute per rowFilter the outer input first; deduplicate keys only when duplicate outer rows are not part of the required result.

Eligible repeatable serial projections reuse complete projected rows using the union of their proven input dependencies. Changing an unused outer column does not force another COUNT evaluation. Changing a returned column remains relevant to the key even if the COUNT itself depends on fewer columns. This is whole-row projection reuse, not independent scalar-subquery caching or grouped decorrelation. Unknown dependency forms (including wildcard subplans and opaque expressions) retain the complete typed input key. The existing row and 8 MiB serialized-data budgets apply; key serialization is bounded and does not clone input values.

Within an eligible projection, filter, aggregation, sort or UNWIND source, each COUNT expression may also cache its own integer result using its proven typed input dependencies. This includes graph patterns and trailing subplans, even when unrelated output columns change. Unknown dependency forms keep full input keys. Reuse is scoped to one operator and snapshot; faults are not cached. Volatile or unproven effects disable it. Each retained scalar entry charges its encoded key plus 264 bytes; all scalar entries and equality tables below share one 8 MiB auxiliary accounting budget and entry cap. Oversized keys and exhausted budgets keep ordinary execution; already retained entries remain usable. Different typed keys still execute the inner plan unless the independent equality-scan path below applies; otherwise mostly-distinct inputs can pay overhead without a benefit.

Boolean wrappers also retain native planned contents: XOR and scalar expressions used as predicates do not hide nested COUNT from these contexts. Predicate values must be Boolean or NULL; other types raise an argument-type error. XOR evaluates both operands once, left to right, while enclosing AND/OR and CASE retain their short-circuit behavior. This does not make volatile expressions reusable.

A snapshot-proven projection, filter, aggregation, sort or UNWIND source may also retain equality counts for COUNT { (n) WHERE n.property = key } expressions (either equality order), including nested arithmetic, functions, CASE, lists/maps, indexing/slicing, list predicates/comprehensions, REDUCE and pattern-comprehension expressions. Here key may be a parameter or a snapshot-stable expression over outer bindings, including node/relationship or map properties, arithmetic and containers. The dependency proof must exclude every inner binding; unknown or volatile expressions retain ordinary execution. Direct id()/elementId() comparison operands retain their special validation/coercion path. On an empty inner relation the key is not evaluated, including later table lookups. On the first nonempty scan its evaluation preserves operand error order; later key errors still propagate through the evaluator. The body may be a single-node scan or a graph path such as COUNT { (n)-[:R]->(m) WHERE n.property = key }. Any named node or relationship in that path may supply the compared property; a proven snapshot-stable inner-only expression such as n.v + 1 or n.v + m.v is also eligible. Its dependencies must be known and contained entirely in the inner path. Arithmetic, functions and containers use the existing expression evaluator in original operand order. Expressions mixing inner and outer bindings, volatile expressions and direct id()/elementId() operands retain ordinary execution. A failed evaluation never populates the retained table. Paths may span multiple fixed or variable-length relationships, including zero-length and unbounded ranges. They use the original traversal and preserve path multiplicity; this does not bound the cost of the first traversal. All named nodes, relationships and internal positional path bindings must be independent of outer bindings, with no repeated names, path binding, optional match, dynamic property constraints, explicit seek hints or trailing clauses. Literal and parameter property filters on nodes and relationships are eligible; the original match iterator still applies them. A leading AND prefix such as n.active AND n.v=key is eligible only when its dependencies are proven inner-only and snapshot-stable. It runs before the equality in the original order: false skips both comparison operands, whereas NULL still evaluates them for errors but contributes no count. A prefix error prevents retaining the table. All-false prefixes also skip lookup-key evaluation on reuse. Trailing, volatile or outer-dependent predicates keep ordinary execution; the planner does not reorder them to force admission. Directions and match multiplicity use the same match iterator as ordinary execution. The first required scan computes its normal answer while building auxiliary counts; subsequent supported scalar keys may reuse them even when other output columns change. Empty outer input does no inner scan. Indexed and seeded patterns retain their existing access paths. Complex inner plans retain their normal execution. Each expression uses its current lexical bindings; no COUNT is evaluated before its branch or loop reaches it. All auxiliary equality tables in one operator share the entry and byte budgets below; this does not grant a fresh budget per nested COUNT. Reused lookups retain cancellation checks within a single row.

Filter contexts cover ordinary WHERE, seeded CALL execution, streaming COUNT predicates and both simple EXISTS predicate loops. Hoisted IN membership retains its prepared candidate set and uses the same context for its COUNT needle. An inner loop owns its context for that invocation; it never shares results with another CALL import or snapshot. EXISTS still stops at its first matching candidate, and short-circuited branches are not evaluated early. Volatile predicates disable reuse. Serial aggregation shares its context across group-key computation, aggregate arguments (including collect with entity preservation), and group-key output. All expressions share one auxiliary budget for that operator, rather than one budget per group. Its existing group and collected-value limits still apply. Parallel aggregation gives each worker its own context on the pinned snapshot; group-key finalization uses a separate context on that same snapshot. Budgets apply per worker, with separate later finalization accounting; 8 MiB is not a global query heap limit. No cache crosses workers or query invocations. Full sort and bounded top-K retain one context across key-computation windows, including seeded CALL sorting. Input and all sort expressions must pass the effect proof; unknown callers retain ordinary evaluation. Stable tie order, NULL ordering and the existing heap/input limits are unchanged. Mostly-distinct keys can cost more; use PROFILE and representative measurements. This is bounded reuse, not general decorrelation.

UNWIND prepares function, list and general expression sources once in the plan. Each source keeps a bounded context across seeds within one repeatable APPLY execution; nested invocations and snapshots do not share it. Literal and plain variable sources retain their existing resolution rules. List entity values, NULL expansion, lazy errors and the existing APPLY/UNWIND row caps are preserved. Procedure arguments and unproven effects retain ordinary evaluation.

The table keeps exact integer keys separate from floating-point comparison buckets, preserving existing mixed numeric equality, including large integers. NULL and NaN do not match. Unsupported inner values or exhausted auxiliary budgets discard the table and retain normal COUNT execution; unsupported outer keys use normal execution. The auxiliary budget charges 256 bytes per retained map entry plus string payload, up to 8 MiB and the aggregation row cap (integer keys can require two entries). This is accounting for retained state, not a measured total query heap bound, and is separate from the projection memo. The first scan incurs grouping work even if no later row reuses it. This is a bounded optimization for an admitted shape, not general decorrelation or a cost-based promise. See the measurement ledger.

Independent COUNT scans with additional repeatable predicates can retain their scan rows when an AND branch contains a proven inner/outer equality. The original complete predicate is evaluated for each outer row in source order; trailing errors, NULL reachability and outer-dependent conditions are not reordered. The first evaluation uses ordinary execution. The second may build a bounded scan cache; later evaluations can avoid graph traversal, but still decode rows and evaluate the filter. This is not a hash join or constant-time count lookup. The cache shares the context’s row/8 MiB auxiliary budget, charges serialized payload plus overhead, and is discarded on overflow or evaluation errors. Volatile or unproven effects, correlated scan bindings and unsupported scan shapes retain ordinary execution. Small input counts may not amortize cache construction; measure representative queries.

Individual repeatable EXISTS predicates also retain bounded Boolean results by proven typed dependencies, so changing unrelated outputs need not repeat them. Unknown dependencies use the full input row. The memo is owned by the same projection, filter, aggregation, sort or UNWIND context as COUNT and shares its entry/byte budget. Cache misses preserve the original edge probe, lazy pattern execution or bounded complex subplan; errors are not stored. Hits still check cancellation, and separate CALL invocations, workers, writes and snapshots retain their existing context boundaries. Distinct keys can add overhead without reducing execution. This is bounded reuse, not a streaming rewrite of complex EXISTS tails.

These are static plan diagnostics, including conditional expression branches; they are not measured cost estimates or automatic rewrites. They appear in plan text, not as a new Bolt notification protocol. Check PROFILE on representative data and preserve zero counts, NULLs, grouping, duplicates and ordering. For example, EXISTS { (n:Absent) RETURN count(*) AS c } is true even when there are no matching nodes: the inner aggregate still returns one row. Removing that aggregate changes the answer. Grouping by an inner property instead returns no row on empty input. A projected NULL still counts as a row for EXISTS; DISTINCT and WITH-level SKIP/LIMIT can remove that row. Complex tails retain reached projection errors even after an earlier row could establish existence; an enclosing unchosen CASE branch still skips the entire subquery. Do not drop those projections or stop early without proving error/effect preservation. A missing warning does not prove that every runtime guard admits an optimized path.

For an independent count reused across rows, a unit CALL expresses computing it once explicitly:

CALL { MATCH (m:Metric) RETURN count(*) AS total }
MATCH (n:Customer)
RETURN n, total

A correlated count is not interchangeable with that query. COUNT/EXISTS bodies accept one pattern and optional WHERE followed by WITH/RETURN stages. DISTINCT, filters, aggregation and LIMIT affect the final row count or existence. These stages retain the normal intermediate-result bounds. Eligible repeatable expressions may reuse results for repeated bindings within one operator. Use CALL for other supported inner-query forms, including UNION. General automatic decorrelation remains tracked in #1230.

CALL subqueries can combine RETURN branches with UNION or UNION ALL, including imported outer bindings. UNION removes duplicates; UNION ALL preserves them. When ORDER BY repeats a projected aggregate, its DISTINCT modifier is part of the aggregate identity: count(x) and count(DISTINCT x) may sort differently. The same identity applies inside scalar sort expressions such as abs(count(DISTINCT x)) or count(x)+1. Sorting by the explicit aggregate alias also selects the intended result. Unaliased aggregate output names such as count(n.value) are treated as exact column names when reused in these expressions.

WITH DISTINCT inside CALL applies before later aggregates, and WITH * can carry bindings alongside explicitly calculated aliases. These paths retain bounded intermediate results; using CALL does not imply automatic decorrelation.

Equality filters inside COUNT, EXISTS and pattern comprehensions can use property indexes, including $parameters and bare outer values (n.key = wanted or wanted = n.key). The supported combination is a conjunction of simple equalities. Complex filters and outer property expressions retain their general path; this avoids changing evaluation errors. The filter is still checked after the index lookup. This improves access per outer row and does not imply that the subquery has been decorrelated or cached across rows.

coalesce() evaluates arguments from left to right and stops at the first non-NULL value. A later subquery fallback is therefore not executed when an earlier value is present. False and empty lists count as present values. Errors in an argument that is actually needed still fail the query.

Correlated relationship bindings

COUNT, EXISTS, pattern comprehensions and CALL retain the identity of an outer relationship reused by name in an inner pattern. OPTIONAL MATCH candidate caching includes bound relationships and resolved scalar seek values. Rejected candidates do not consume the materialized-row limit, but still count toward query deadline checks. A relationship identity filter can still require scanning candidate edges; it does not imply general decorrelation or a direct edge seek.

A sorted scan inside correlated CALL also honors SKIP, including parameterized offsets. It applies the offset after binding validation and before bounding the collected rows. This path may visit the skipped candidates, unlike standalone key-only index pagination; an index does not guarantee identical pagination cost inside and outside CALL.

Bounded reuse of repeated CALL inputs

Within one APPLY execution and storage snapshot, deterministic read plans can reuse the result for identical declared CALL imports. Other outer columns do not enter a CALL scope or its memo key. CALL without imports has an empty seed and can reuse one result when outer rows exist. Internal planner APPLY retains the complete outer row as its dependency key. Outer duplicates and ordering remain present in the output; empty inner results remain empty. Keys preserve runtime types and representations, so Integer and Float inputs do not share a memo entry merely because Cypher considers their values equal.

The memo holds at most the configured row limit in entries and result rows, and at most 8 MiB of encoded keys/results. Oversized inputs or a full memo keep ordinary execution. Query deadlines, output limits and evaluation errors still apply. Each APPLY invocation owns its memo; it is not reused across queries, parameter changes or transactions.

Admission uses a snapshot-local effect analysis, separate from cross-transaction cache invalidation rules. It recursively covers read-only COUNT/EXISTS, list predicates/comprehensions, CASE, REDUCE, OPTIONAL MATCH, UNION and nested CALL, including unlabeled nodes and untyped relationships. Executable pattern and seek expressions are checked too. Procedures, writes, external input, volatile functions and unproved expression shapes remain excluded. Shortest-path escape hatches currently retain ordinary execution. An explicit non-wildcard output projection allows proven unused imports to be omitted from the memo key. Unknown dependencies, wildcard output and unsealed output retain all imports. Different required imported values still execute separately; this is not general grouped decorrelation. EXPLAIN/PROFILE identifies that remaining work and suggests filtering the outer input or a semantic-preserving grouped rewrite where appropriate.

When APPLY reuse is excluded, additional plan diagnostics identify the cause:

DiagnosticCause and suggested action
PDB1230-VOLATILERandom/clock-dependent evaluation; bind once outside CALL only if one value per query is intended.
PDB1230-PROCEDUREProcedure effects are unproved; batching or deduplication must preserve invocation effects.
PDB1230-EXTERNALExternal input may change; read once only if a single input snapshot is intended.
PDB1230-MUTATIONWrites cannot reuse prior results; reducing duplicate writes changes semantics unless intended.
PDB1230-PROOFUnsupported proof shape; retain per-row semantics and use PROFILE before a grouped rewrite.

These messages do not recommend silently replacing per-row randomness, external reads or writes with a once-only operation.

Aggregate expressions in lists, maps, index/slice expressions and CASE conditions are evaluated at their enclosing WITH or RETURN grouping stage. For example, UNWIND [1,2,3] AS x RETURN [count(*)][0] AS c returns one row with c = 3. Aggregates within an inner COUNT/EXISTS query keep the inner scope.

Aggregates in iterator sources belong to that enclosing grouping stage too: UNWIND [1,2,3] AS x RETURN size([y IN collect(x) | y]) AS n returns n = 3. REDUCE may consume an aggregate list and use an aggregated initial value. Local iterator variables are available in the body; other outer references must be valid grouping keys when the expression aggregates.

Comprehensions, REDUCE and ANY/ALL/NONE/SINGLE accept the same scalar source expressions, including direct list parameters, slices, CASE, nested comprehensions and COALESCE. For example, [x IN $items[0..10] | x] does not require an intermediate WITH solely to name its source. NULL and empty-list behavior is unchanged.

Introspection functions evaluate their argument first: keys(box.value) and properties(box.value) inspect the selected value, not box. Entity-valued fields retain their node/relationship identity for id, elementId, labels, type, startNode and endNode.

Runtime argument errors inside iterator bodies fail the query. type() accepts a relationship or NULL; a node or other non-relationship is an argument error. Strict toBoolean/toInteger/toFloat/toString reject list, map and graph-entity/path arguments. Use the corresponding ...OrNull function when NULL on unsupported types is intended. This is distinct from an unparseable conversion string, which can still return NULL.

Inline aggregates in iterator bodies (for example [x IN [1] | count(*)]) are a planning error, including when the input is empty. Compute the aggregate in a preceding WITH and reference its value in the iterator, or use an explicit COUNT/EXISTS subquery with its own grouping scope. Aggregate sources such as [x IN collect(n) | x] remain supported.

Within an aggregating CASE expression, every reference outside aggregate arguments must be a projected grouping key. This applies to WHEN conditions, THEN/ELSE values and simple CASE scrutinees. For example, project x alongside CASE WHEN count(*) > 0 THEN x ELSE 0 END to group by x; without that grouping key the query raises AmbiguousAggregationExpression.

Grouping aliases preserve the input variable’s value inside correlated COUNT/ EXISTS queries, pattern comprehensions, paths and boolean predicates. Local iterator variables retain lexical shadowing. Internal group/aggregate names are chosen to avoid user identifiers; using an alias that matches another input or local variable does not change that variable’s value.

A property-only grouping key such as n.v also retains that property inside correlated scalar subqueries. It does not retain the whole n or unrelated properties: using those in the same aggregating expression raises AmbiguousAggregationExpression. Project the additional grouping key only if its finer grouping is intended; otherwise compute the needed value before the aggregation. Grouping by the whole entity retains its identity and properties.

Dynamic properties inside scalar patterns

COUNT, EXISTS and pattern comprehensions evaluate dynamic pattern properties against the current row, including bindings, property accesses, arithmetic and functions. This also applies when COUNT or EXISTS has a WITH/RETURN tail. Anonymous elements remain anonymous in wildcard output and DISTINCT semantics. For a variable-length relationship, dynamic property conditions apply to every relationship in the path; a zero-length path satisfies that condition vacuously. Unreached CASE branches and empty candidate scans do not evaluate the property expression. Reached evaluation errors propagate. These correctness rules do not imply that a dynamic property pattern is eligible for grouped COUNT reuse.

Dynamic pattern properties

Dynamic node and relationship properties in MATCH are evaluated against the current row, including anonymous patterns such as MATCH (:Item {value: k}). Their internal bindings do not appear in wildcard results or change DISTINCT row identity. In OPTIONAL MATCH these conditions belong to the optional clause: when no candidate satisfies them, the incoming row remains. Variable-length relationship conditions apply to every edge; a zero-edge path has no edge condition to evaluate. As with an explicit WHERE expression, a reached invalid expression reports an error; absent candidates do not evaluate that expression.

RETURN DISTINCT * exposes the bindings still visible after the preceding clauses. ORDER BY can refer to those bindings and their properties, including within scalar subquery expressions. A name removed by WITH or never introduced is still unavailable; an explicit DISTINCT projection retains only its projected columns for ordering.

Filters before later reading clauses

A required MATCH ... WHERE filter is applied before a following MATCH, OPTIONAL MATCH, UNWIND or CALL. Dynamic pattern properties obey the same boundary. A row discarded by the required clause does not evaluate later expressions; an error reached by a surviving row remains an error. Patterns separated by commas within one MATCH are evaluated together before its filter.

A literal equality selecting a single anchor before a separate traversal MATCH can still use a covering edge-sorted index for top-K and receive its index recommendation. Extra anchor predicates and unproven shapes retain ordinary execution; multiple matching anchors retain the existing fallback.

OPTIONAL followed by another reading clause

A following MATCH, UNWIND or CALL consumes the rows produced by the preceding OPTIONAL MATCH, including its NULL bindings. Required matches cannot bind those variables before the optional clause applies its own predicate. A missing optional node followed by a required relationship from that node produces no rows; an independent required pattern can still combine with the NULL row.

Reading order before MATCH

A preceding UNWIND or CALL executes before the subsequent MATCH. For example, UNWIND [1/0] AS k MATCH (n:Missing) RETURN k reports the arithmetic error even when the later label has no matches. Empty prefixes still produce no downstream rows, and duplicate prefix rows retain their multiplicity. Explicit WITH is not required to preserve this boundary.

OPTIONAL clauses with multiple patterns

Comma-separated patterns inside one OPTIONAL MATCH must all match and satisfy the clause’s WHERE. Otherwise the input row survives once, with all newly introduced bindings set to NULL; existing outer bindings retain their values. Two separate OPTIONAL MATCH clauses are evaluated independently in order.

Relationship identity across patterns

Within one MATCH or OPTIONAL MATCH clause, different relationship occurrences must use distinct physical relationships, including across comma-separated patterns, anonymous relationships, named paths and variable-length paths. Separate MATCH clauses may reuse a relationship. A zero-length path contributes no relationships to this check. An OPTIONAL clause without a complete valid match retains its incoming row with new bindings set to NULL.

Internal bindings used for these checks are not exposed by wildcard projections. Repeated occurrences of the same node variable, such as a self-loop’s endpoints, appear once in RETURN * and WITH *, including DISTINCT projections. The existing intermediate row and cartesian-product limits still apply.

The relationship uniqueness check completes at the MATCH clause boundary, before subsequent UNWIND, CALL or OPTIONAL MATCH clauses run. Rows rejected by that check do not evaluate expressions in those later clauses. Errors on valid matching rows remain observable.

Automatic integer covering input (#1399)

A broad plain single-label aggregate reading one integer property automatically uses a covering value-index scan when the shape, row estimate and scratch budget pass admission. Stores/inputs estimated below 4096 rows retain existing access. Configured parallel workers also retain existing aggregate admission; this initial policy does not compare covering against parallel throughput. A bounded external merge restores the original UUID order before the existing evaluator consumes values. This avoids changing SUM intermediate overflow, floating-point expression accumulation, group encounter order or DISTINCT behavior merely because an index sorts by value. Missing properties retain NULL semantics. Original group/buffer limits still apply.

The candidate declines floats (index encoding canonicalizes signed zero), wide values, secondary labels, pending derived indexes, copy-on-read mirrors, selective patterns and unproved expressions. Existing grouped-count index execution keeps precedence. EXPLAIN exposes the selected candidate or its fallback reason, including estimated reorder work and insufficient scratch. Encoded key bytes exclude page overhead; current value-index payloads are empty, and primary I/O costs remain unknown. A logical byte estimate is not a prediction of physical reads. Limited synthetic memory-pressure qualification covers this narrow increment; cost selection remains heuristic and broader predicate/type coverage is follow-up work. PATINADB_READ_COVERING=off disables covering for diagnosis. See the configuration controls.

EXPLAIN describes the proposed access without running the query. PROFILE keeps that pre-execution plan and appends Executed aggregate access (query thread), collected at runtime. For example, eligible execution reports agg.covering_integer_uuid with the actual index-record count and reorder=memory or reorder=external merge. Retained execution reports a covering-decline reason and agg.existing_input (serial aggregate) or (parallel aggregate). Existing grouped/count fast paths report their own agg.* name.

The planned choice can differ from execution: an underestimated scratch budget or unavailable scratch during preparation falls back before emitting rows. PROFILE reports that fallback rather than claiming the proposed covering path ran. Index/storage errors, cancellation and duplicate/orphan identities are not converted into successful results. Diagnostics record at most 64 aggregate events, followed by an omitted-event count; they are not per-worker or per-row timings. For a query without observed aggregation the output says so explicitly. elapsed_ms still measures execution, including abandoned preparation, not EXPLAIN planning.

Bounded preparation for filtered aggregates (#1399)

Eligible residual string predicates can obtain candidate node IDs from the value index before aggregation. This preparation is chosen automatically; existing selective seeks keep precedence. It is separate from the integer covering input and its diagnostic switch.

The filter first reuses a prefix of at most 64 decoded records or 64 KiB of encoded keys, with one final key allowed to cross the byte threshold. Already cached column population and observed key width can predict preparation that would exceed its 64 MiB logical key-work budget. The policy never rebuilds a full statistics catalog just to decide admission. A value-sorted prefix is biased, so its match rate is extrapolated only when the first and last encoded values prove that the column is uniform. Missing column statistics retain runtime bounds; label cardinality is not substituted for property population. Stale estimates can change the selected path but do not supply query results.

For supported COUNT grouping, a small exact set of selected roots can keep ordinary property reads instead of preparing the full grouping column. The current heuristic applies to at most 16 selected roots, an estimated label with at least 1,024 nodes and a ratio greater than 64:1. A bounded width prefix can also reject estimated group preparation exceeding the existing 16 MiB retained-value budget. These thresholds are admission heuristics, not a calibrated I/O price.

EXPLAIN reports a planned bounded prefix probe, available advisory counts and budgets. It does not evaluate the predicate to measure selectivity. PROFILE’s executed-access section reports filter.covered_candidates and, when attempted, filter.group_values, with the selected path or fallback reason. Record/key-byte counters cover consumed preparation entries; they exclude metadata/domain probes, page metadata and physical I/O. Worker-local preparation is not included in the query-thread capture. Copy-on-read overlays and pending indexes retain existing access before creating the covering iterator.

Runtime candidate, retained-value, byte and deadline limits remain active even when estimates are too low. One large encoded value must still be read to inspect it; these bounds do not promise total process RSS. The policy can discard a bounded amount of preparation before fallback and does not guarantee that every filtered query becomes faster.

Inline and external vertex properties (#1399)

The logical primary key still begins with the owner UUID, preserving per-owner prefix reads. Schema 7 keeps canonical values up to 256 serialized bytes inline and places wider canonical values in a property/owner payload table. The owner entry carries a tag. Ordinary point reads and hydration resolve that tag using the same storage snapshot; writes update both tables in one redb transaction. This routing is automatic and has no activation flag. The threshold is a fixed layout heuristic, not a measured optimal boundary for every workload.

Keeping small values apart from wide neighboring values helps workloads that need a small property through primary access. Existing costed index/covering, selective and aggregate choices remain in place. The evaluator still decides which values to read and in which order. A property-ordered scan is not selected: its prototype did not demonstrate enough benefit to justify another access path. Broad reads of the wide values themselves are not guaranteed to improve; warm point/hydration overhead and write/index costs still matter.

EXPLAIN’s Vertex property storage line describes the physical representation, separately from the query operators and storage I/O policy. PROFILE adds Resolved vertex property storage: actual inline/external resolutions and their canonical byte counts on the query thread. These include point and range reads; worker threads, caches and buffered values may bypass them. They are not disk traffic, evaluator-call counts or a total-memory measurement.

The payload table retains canonical bytes, including nested values and exact floating-point representations. Missing external data fails as a storage error; it is not interpreted as a missing property. Value-index keys keep their full ordered value, and repeated wide values are not interned across owners. Schema 6 stores require recreation/reimport; no on-disk compatibility migration or dual decoder is included in this format change.

Wide value-index keys (#1399)

Value indexes keep the encoded property value in the key so equality, prefix, range and ordered scans retain their existing semantics. The authoritative property tree keeps the original value. The secondary payload is empty for both direct setters and batch/rebuild paths.

redb 4.3.0 abbreviates byte-key separators in internal B-tree pages where the neighboring keys permit it. Full keys remain in leaf pages: this is neither string interning nor removal of repeated leaf-key bytes. Long common prefixes can prevent useful shortening, and wide full-index scans still visit wide leaf data. This backend representation is automatic; EXPLAIN and PROFILE keep reporting the same logical access paths. No planner flag enables it.

Smaller indexes do not guarantee fewer physical reads for every access path. Constrained-memory qualification found primary scans that read more from disk with the new file layout, despite similar requested read volume. Account for that tradeoff when measuring a workload dominated by wide primary properties.

The final wide-key investigation retained this representation. A fixed-prefix multimap can save space but may scan a whole suffix group for a selective lookup or late page. A separate prefix dictionary restores suffix seeks, yet adds indirection that regressed distinct-wide reads in the native controls. Grouping complete identical values benefits duplicate-heavy data but does not shorten unique values. These experimental representations are not product options.

The source repository records the method and counterexamples in docs/wide-index-layout-decision-1399.md; earlier access-path work is in docs/read-path-1399-plan.md. This concludes the bounded optimization round, without claiming customer/TB qualification or universal optimality.

Primary-property I/O under memory pressure (#1399)

Logical access and aggregate ordering are unchanged. For an existing Linux store larger in allocated bytes than detected RAM, pure primary-property point reads can use a separate no-read-ahead file handle. This avoids applying the same hint to index/range scans or writes. The policy is selected at open and is reported by EXPLAIN’s Storage: line. Smaller stores and unsupported/unavailable platform facilities retain native I/O; PATINADB_PRIMARY_POINT_IO=off is a diagnostic override requiring reopen.

PROFILE’s Storage I/O footer distinguishes backend requests through the primary handle from native requests on the query thread. It excludes worker threads and redb cache hits and does not measure physical disk bytes. Cold latency can still trade off against speculative reads; this is not a universal scan speedup or a replacement for workload-specific measurements. The counters instrument the Linux file backend only; memory, simulation and other platform backends are uninstrumented, rather than evidence of zero I/O.

Fixed traversal to an already bound node

A fixed, single-type hop whose destination has UUID equality can test the typed edge key directly. Directed and undirected relationship semantics, edge properties, destination labels/properties and relationship uniqueness still apply. Other traversal shapes retain the general iterator. This rule has no configuration option; a speedup for a complete query is not yet established by the current LSQB work (PR #1410).

UUID point entry access also completes at the selected vertex: required labels are checked against that vertex’s primary and secondary labels, followed by all remaining property predicates. It does not append the general secondary label scan. Missing vertices yield no row; errors reading the candidate’s own labels or properties remain errors. Other entry access methods retain their existing candidate enumeration (PR #1410).

For property-free targets, traversal can reuse the primary-label ID already stored in the edge key, including targets without a label constraint and secondary-label checks. Secondary membership still has to be read and validated; this only removes a redundant primary-label lookup. Cached edges without an inline ID and property/multi-label constraints retain the full validation path. The existing optional liveness check remains in effect. No new setting or whole-query speedup guarantee follows from this access refinement.

Counting independent path branches

For eligible count(*) aggregates, agg.disjoint_path_count counts each side of a fixed path around a pivot and multiplies the counts. It avoids constructing every combination of the two sides. Each branch streams; count arithmetic checks for overflow and execution continues to enforce the query deadline.

Explicitly configured parallel query workers retain existing parallel admission. Eligibility requires one required path, no grouping or DISTINCT, and disjoint relationship types across the split. Binding names may not repeat. Optional or variable-length paths, property/seek constraints and path bindings retain the existing execution. A proven filter may move to one factor when every referenced binding is a vertex of that factor. Supported expressions are vertex = / <>, AND, NOT, and bare EXISTS for one fixed typed edge between bound vertices. The EXISTS form permits at most one label constraint per endpoint, but no local bindings, properties, seeks, named edges or additional scope. Cross-factor predicates, functions and other subquery forms retain the existing execution.

For the newly admitted compound or EXISTS predicates, the opposite factor is counted first. If it is empty, the predicate is not evaluated: there are no joined rows on which to evaluate it. Otherwise predicate evaluation retains ordinary bindings and fallible errors propagate immediately, without replay or fallback. Count products and sums remain checked; deadlines and existing limits remain. No isolated speedup is claimed for this extension.

For an eligible pure count factor with a fixed endpoint, typed single-edge EXISTS checks may reuse a neighbor set scoped to that factor. A complete set can prove absence; possible matches still use the ordinary point check. Duplicate paths retain their multiplicities. Set memory is charged to the query budget in addition to the existing pair memo, with a separate bounded scan cap. An incomplete capped scan is discarded and ordinary probes continue. Memory refusal, cancellation and read errors propagate without retry. Only live index views admit this optimization; overlay views retain ordinary probes. No new configuration option is introduced (PR #1410).

EXPLAIN reports the selected pivot as a path-element offset; PROFILE confirms actual use. Selection balances branch lengths without a graph-wide statistics scan. This is a heuristic, not a universal cheapest-plan guarantee. The original LSQB comparison at 928cd9ad completes Q1/Q6 with exact counts; other query regressions remain. This is not general performance qualification (PR #1410).

The development extension agg.grouped_disjoint_path_count applies the same factorization to one vertex grouping key at the pivot and one count(*). It retains the ordinary full-path entry choice, traverses that entire half, and counts its rows by the grouped vertex using the existing bounded aggregate. The opposite half is counted once per source group and its count is multiplied with the source multiplicity using checked arithmetic. Only the pivot binding is retained from each source row; duplicate paths still contribute to the count. Groups without an opposite match are removed before the eager result is exposed.

Existing group/output row bounds remain in force. Intermediate source groups can include vertices with no opposite match, so this strategy can refuse at the group cap even when the final group set would fit. It does not raise the cap or retry through another strategy after a resource error. Filters, scalar or multiple grouping keys, DISTINCT, optional paths and unproved sources retain the ordinary executor. Result order without ORDER BY may change; explicit sorting and limits retain their operators. PROFILE records the chosen entry and source-side grouping. The source-entry correction has focused correctness and work-count controls; no isolated speedup is claimed.

Eligible predicate-free COUNT branches reuse the ordinary traversal state machine while omitting terminal row payloads. Candidate checks, relationship uniqueness, label validation and deadlines still run for each match. Branches that evaluate a row predicate retain their bindings. Repeated bindings, middle-entry products, variable-length paths, optional/path bindings, property/seek constraints and active join guards retain ordinary row emission. This changes materialization, not cardinality or traversal semantics; no configuration toggle is added.

A development COUNT adapter can also compare two uniquely bound vertices for = or <> directly from their UUIDs during successful path emission. It uses the same traversal, target validation and relationship-uniqueness checks, and counts complete matches with the existing overflow checks. Unsupported bindings or expressions retain row evaluation before consuming the iterator. This avoids rows for that proven predicate; it does not skip graph validation or prune paths. Focused and restored Q6 controls plus three genuine mutations passed. The later PROFILE validation also passed the retained path-count tests after adaptive removal. Combined broad source validation passed (PR #1410). Ordinary-binary comparisons of the combined changes are recorded in the LSQB measurement ledger; those bundle measurements do not isolate this adapter’s contribution.

A development scalar COUNT alternative can choose an entry within the shorter independent factor when its primary-label cost estimate is smaller than the original pivot’s. It reuses the grouped admission proof, source_half split and shared traversal, then combines each pivot’s count with its opposite branch using checked arithmetic. Exactly one existing MemoryHashMap<Uuid, i64> retains the counts; its actual table allocation and growth consume statement memory credits. The existing aggregate group cap remains. No representative-row collection or second key set is retained. For a uniquely bound stable vertex, the source factor can emit its UUID directly through the same completed traversal rather than build a binding row only to extract that UUID. Admission requires the existing endpoint/fixed-path proof; unsupported shapes retain row evaluation before consuming input. Match order, duplicates and visited errors are preserved. This later source-emission change passed focused/restored and mutation controls. The later b338 combined-binary comparison includes this change; it does not isolate the gain from UUID emission.

Legacy calls without a statement memory scope retain the streaming path. Group bindings _, _anon and internal property bindings also decline before execution. Once execution starts, visited errors, cancellation and resource refusals propagate without replay. The additional retained groups can cause a memory refusal even where the streaming alternative would have completed.

Primary-label counts do not describe the full secondary-label population and are used only as cost heuristics. Cold planning uses bounded index probes, adding planning work; this does not prove a better traversal order. EXPLAIN shows the enclosing candidate, while successful PROFILE execution records agg.local_anchor_path_count with the chosen pivot and entry. Focused tests, the actual hook-bypass counterexample and restored-source lint checks passed. Combined broad source validation and the b338 ordinary-binary comparison are recorded in the measurement ledger. The comparison is descriptive, uses shared OS cache and includes other changes; it does not establish a universal benefit or performance under RAM pressure (PR #1410).

The experimental bounded suffix-count memo was removed after its ordinary-binary comparison established no benefit. Scalar counts retain direct branch traversal and the general row-elision paths described above. Historical measurements are recorded in docs/lsqb-sf01-coverage-2026-10-11.md; the experiment is not a current execution strategy or configuration option.

Grouping consumes owned evaluated keys without copying their heap-backed payload a second time. Borrowed callers retain copying behavior. Numeric equivalence, NULL grouping, entity identity and first-occurrence output representatives remain unchanged. This removes one copy; it does not make grouping allocation-free or remove its existing memory and group limits. No isolated speedup is claimed for these materialization changes.

Closed paths and endpoint joins

The development planner preserves a join alternative for two fixed paths sharing both distinct endpoint bindings when each relationship has one explicit type and the type sets do not overlap. The normal join optimizer still chooses execution order and hash versus seeded execution. Clause-wide relationship identity checks remain. Open chains and shared, untyped or variable relationship domains keep the existing fold policy. The original LSQB Q2 comparison at 83002192 still selects seeded execution and times out. Preserving a join alternative alone does not establish improved performance (PR #1410).

The single-probe cost comparison no longer selects repeated seeded execution for multi-hop paths with unbound interior vertices: it does not estimate the intermediate traversal work. Such cases retain hash execution unless the separate policy for unknown anchored paths applies. One-edge and fully bound interior paths retain their existing cost comparison. Performance validation of this development refinement is pending (PR #1410).

Streaming APPLY and partial join counts

APPLY streams its results one outer seed at a time when the snapshot-read analysis proves the entire consumer plan repeatable. Recursive iterator construction inherits that decision; unproven procedures inside scalar COUNT/EXISTS keep the eager read boundary. The outer input and each seed’s result are still collected, and nested consumers that require rows still collect them. Cumulative output limits and deadlines remain. Unproven procedures, mutations, external input and volatile expressions keep eager evaluation, including before a downstream LIMIT or early iterator drop. A downstream LIMIT or early drop can avoid evaluating later pure-read seeds; errors or cap breaches confined to those unconsumed seeds are therefore not reported. Errors encountered in the consumed prefix still propagate. This is lazy read evaluation, not preservation of the former eager error timing.

After ordinary storage-aware join ordering, eligible ungrouped count(*) over an inner join can group both inputs by their shared vertex keys, join those counts, and sum their products. This preserves multiplicities while avoiding materializing the corresponding cross-product rows. Scalar keys, DISTINCT, optional matches, user grouping, intervening filters and unproved source plans are excluded. A selected APPLY plan is retained. Existing group and memory caps apply; changed physical retention can change when resource refusal occurs.

Relationship identity guards between comma-separated patterns are omitted when their explicit relationship type domains are disjoint. Overlapping alternatives and wildcard types keep their guards. No graph label names or benchmark query texts select these rules.

Exact join membership during fixed-path traversal

For structurally pure keyed inner joins over a correlated fixed path, the executor can reuse the existing join hash table to reject vertices that cannot contribute a result before expanding their remaining path. The original join still emits every matching row, preserving duplicate witnesses. This requires proven vertex bindings and ordinary variable identity; anonymous internal names, scalar keys, optional paths, variable-length paths and predicate/effectful boundaries retain ordinary execution. No graph label or benchmark name selects this optimization.

The filter shares the join table instead of building another set of keys. Additional fixed control structures are charged to the query memory scope; the existing join-table accounting remains unchanged. At most eight keys and eight filters are admitted; larger shapes use the ordinary executor. There is no configuration toggle. Deadline and memory limits are unchanged. Work pruned before expansion performs no downstream reads, so errors in those unvisited branches are not evaluated. Errors from consumed storage reads still propagate.

The extension join.grouped_identity_membership also checks an identity vertex grouping key between two eligible grouped COUNT(*) inputs. Both inputs must satisfy the fixed-path grouped-count proof, and the inner join must use their single output key. Output aliases are supported; anonymous/internal bindings, inherited membership filters and effectful consumers retain ordinary execution. It borrows the already built right join table while evaluating the left aggregate, after source-side grouping and before counting each group’s opposite branch. Source rows and group buffers are still constructed; only the unmatched opposite-branch work is saved. Counts for retained groups and the original join’s multiplicity are unchanged. A deadline check precedes every membership decision, including empty right inputs. Encountered source, allocation and traversal errors propagate; skipped opposite branches do not evaluate potential storage errors or count overflow.

Fixed-hop storage errors

Fixed-hop traversal preserves upstream adjacency storage and decoding errors through relationship-label and property filters. Such an error fails the query rather than silently removing the affected row. This also applies when grouped COUNT uses the same traversal. Existing property-lookup and missing-target handling are unchanged.

Secondary-label membership scans

Single-label scans of secondary labels avoid temporary decoded ID/label vectors when only membership is needed. Each visited record still validates all label IDs, even after a match, and preserves primary/secondary deduplication. Multi-label scans keep the existing path.

An experiment that propagated required labels between MATCH patterns before plan construction was withdrawn after a measured planning regression (PR #1410); the anchored-cycle policy before it is in effect.

Direct EXISTS for a bound typed edge

An EXISTS over a single fixed typed edge between two uniquely bound vertices uses a direct index seek. It preserves endpoint/label checks, direction and errors from performed reads; unsupported shapes use ordinary execution. The seek may change which unrelated records are visited. Key construction reuses the target primary label just read and validated in the same transaction; it adds no cache and assumes nothing from a label carried in an input row. No setting is added. In the LSQB comparison of this change together with the planner rollback, Q3 recovered while Q8 and Q9 kept their 8 s timeouts; no isolated speedup is claimed. See the LSQB measurement ledger.

Completed-prefix filter pruning

For a pure fixed path with unique bindings, a leading AND condition may depend only on a prefix of the chosen endpoint traversal. After the first complete row has passed the normal path consistency checks, a false leading condition can skip the remaining suffixes of that same prefix. Duplicate prefix visits and physical-edge uniqueness retain the ordinary iterator semantics. UNKNOWN still evaluates the right condition; performed-read and evaluation errors propagate. An empty suffix never causes an earlier predicate evaluation. Pruned unread suffixes need not expose errors that ordinary expansion would have encountered.

Optional or variable paths, repeated bindings, seek hints, inherited join guards, and unproven effectful consumers retain ordinary execution. A middle entry or a condition depending on the final leaf retains the ordinary iteration on the same constructed source. No query name, graph schema, new cache or forced entry order selects this optimization.

Shared mandatory labels and branch access

After selecting ordinary join order and Hash/Apply strategy, the planner can propagate an unambiguous label constraint between occurrences of the same joined vertex within one mandatory MATCH scope. It then refreshes leaf scan strategies before eligible partial COUNT grouping. The rule preserves the selected join tree and does not cross optional matches, dependent predicates, scope rebinding or writes. It has no separate configuration switch.

Label population is only an entry estimate; fewer starting vertices can still produce more traversal work, so this rule does not guarantee a faster plan. Separate MATCH clauses and unsupported scopes retain their existing access planning.

Lazy compound IN membership

A sole immutable parameter or literal IN operand may use a lazily constructed, operator-local membership set when execution first reaches that predicate. AND, OR, XOR and NOT retain the ordinary recursive evaluator: skipped branches build nothing, and lhs/null and sticky-error handling precede membership lookup. Only stable plan predicates participate. Safe scalar candidates reuse the existing numeric-equivalence guard; collection needles or unsupported candidate values retain linear three-valued membership. Row-dependent RHS evaluation is unchanged.

Hash tables and retained keys use the existing query memory admission APIs. Total admitted candidate entries are bounded per operator; admission failure is propagated without replay, and context destruction releases retained owners. This introduces no shared result cache or new configuration flag. No isolated speedup is claimed.

Factor consumers

With PATINADB_FACTORIZE on (the default since 2026-10-17, #1442; off restores the ordinary operators), three more operators can work on a pattern that splits at an interior pivot into two independent branches (RFC 0042 B3–B5, #1442). They read each branch once per pivot instead of building every combination:

  • Top-k within a factor. ORDER BY … [SKIP s] LIMIT k, when each sort key belongs to the pivot or to one branch.
    • Each branch is sorted once per pivot, and only combinations that can still enter the page are expanded.
    • The edge-sorted and ordered-frontier paths keep priority.
    • If the page contains tied keys on different rows, the ordinary bounded top-k runs instead, so tie order is unchanged.
  • DISTINCT over one factor. RETURN DISTINCT under ORDER BY (with or without LIMIT), and WITH DISTINCT … feeding an ungrouped count, when the projection reads the pivot and at most one branch. The other branch only has to exist, so it stops at its first matching row.
  • Counts with a cross-branch comparison. Ungrouped count(*) with one a <> b, NOT a = b or a.p <> b.q (or =) between the branches is counted per pivot by inclusion–exclusion.

A branch with a single element at a pivot is folded into the pivot (flatten). When the factors stay degenerate, for example a single hop from an anchored node, the whole operator runs the ordinary plan instead. That choice is made from the operator’s own input and never changes results (ADR 0019).

PROFILE lists the executed choices under factor.topk, agg.factorized, factor.exists and factor.flatten. EXPLAIN is unchanged.

They have no cost admission: every proven shape engages, with or without the cost model. See limitations.

Factor-local grouping

PATINADB_FACTORIZE (on by default since 2026-10-17, #1442) enables a grouped aggregate for proven independent fixed branches. For two branches around a pivot vertex it reads each branch’s group keys once per pivot and combines the key multisets directly into the group table; the existing ORDER BY and LIMIT still apply afterward. PROFILE records agg.factorized when this path executes. It requires an admitted read memory scope. Unsupported patterns or value domains use the reference aggregate, and factor memory refusal falls back before output.

Branches may share a relationship type when they reach the pivot through it in the same way, for example two comments of one ticket:

MATCH (u1:User)-[:AUTHORED]->(:Comment)<-[:HAS_COMMENT]-(t:Ticket)
      -[:HAS_COMMENT]->(:Comment)<-[:AUTHORED]-(u2:User)
WHERE u1.name < u2.name
RETURN u1.name, u2.name, count(DISTINCT t)

Relationship uniqueness then requires two different comments. The engine checks this per ticket at run time, also when a comment has several authors. Floating-point aggregation and SUM with overflowing positive/negative subtotals are not currently admitted.

With PATINADB_COST_MODEL set to entry or join (the default join), the grouped path engages only when the degree catalog predicts that it is cheaper than the flat aggregate; without a catalog the flat aggregate runs. The catalog comes from the background sample the cost model requests on a miss, or from CALL patinadb.stats.degrees(). With PATINADB_COST_MODEL=off every proven shape engages. The weights are fitted on one benchmark host, so treat them as indicative; see RFC 0042 B1/B2 (#1442).