Skip to main content

Architecture

Gnarl replaces the cluster with a network: a set of independent nodes that each own their data, their index, and their policy, and that cooperate only for the duration of a single query.

The pieces​

        ┌──────────────┐        ┌──────────────┐        ┌──────────────┐
        │   node A     │        │   node B     │        │   node C     │
        │  ──────────  │        │  ──────────  │        │  ──────────  │
        │  local data  │        │  local data  │        │  local data  │
        │  local index │        │  local index │        │  local index │
        │  policy      │        │  policy      │        │  policy      │
        └──────┬───────┘        └──────┬───────┘        └──────┬───────┘
               │  manifest             │  manifest             │  manifest
               └───────────────┬───────┴───────────────┬───────┘
                               │                       │
                        ╔══════╧═══════════════════════╧══════╗
                        ║        the network (gossip)         ║
                        ╚══════════════════╤══════════════════╝
                                           │
                                     query enters
                                     at any node

There is no node in that diagram that is more important than the others. The node a query happens to arrive at becomes the coordinator for that query only, and stops being special the moment it returns results.

Query lifecycle​

  1. Parse. The receiving node parses the query and resolves it against the fields it knows exist across the network.
  2. Plan. It reads cached manifests and eliminates every node that provably cannot contribute — wrong field, wrong time range, wrong geography, wrong policy. Planning is local and takes microseconds.
  3. Fan out. The query is signed and sent to the surviving candidates, with a deadline attached.
  4. Execute locally. Each node runs the query against its own index, applies its own policy, and returns hits with a signature.
  5. Merge. The coordinator normalizes scores, merges the ranked lists, and streams results back as they arrive.
  6. Verify. Signatures are checked against known node identities. Unverifiable results are dropped and reported, not silently included.

What is deliberately absent​

AbsentWhy
Elected masterNothing to elect, nothing to lose during an election
Shared cluster stateNodes agree on nothing except the protocol
Global rebalancingData placement is whatever the operator already chose
Cross-node transactionsQueries are read-only and independent
Central registryManifests gossip; there is nobody to take offline

Failure behavior​

A node that is slow, unreachable, or refusing is not an outage — it is one fewer answer. Every response carries the roster of who was asked and who replied, so a partial result is always visibly partial:

{
  "hits": [ /* ... */ ],
  "coverage": { "asked": 41, "answered": 39, "timed_out": 2, "complete": false }
}

Applications that need completeness can require coverage.complete, retry the stragglers, or degrade explicitly. What they cannot do is mistake a partial answer for a whole one.

Consistency model​

Gnarl is eventually consistent at the manifest layer and strongly consistent at the node layer. A node always answers from its own committed index. What varies is how quickly other nodes learn that the index changed — typically a gossip round, on the order of seconds.

See also​