A St. Bernard finds what is buried. Not in a controlled environment, not with precise coordinates — in bad weather, across terrain nobody mapped, working from a faint signal and its own judgment. Then it brings back something useful.

That is the whole product description.

Centralized search is the opposite metaphor. It says: before we can find anything, move the mountain here. Sort it into our bins. Then we will search it, quickly, in ideal conditions.

That works right up until the mountain is too heavy to move — or legally cannot be moved at all. Which, increasingly, is the normal case.

The barrel​

The barrel carries the Gnarl knot: an unbroken loop with no beginning and no center. That is the topology. There is no head node in a knot.

It is also, historically, a myth — St. Bernards never actually carried brandy barrels; a painter added one in 1820 and it stuck. We kept it anyway, because the useful part of the image is that the dog arrives carrying something, not what is in the barrel.

The cover​

There is a second reason. The St. Bernard on the cover of Scaling Search and Retrieval for Contextual AI is the same dog. O'Reilly picks the animal; the author does not. Getting a search rescue dog for a book about finding things in enormous piles was luck, and it seemed wasteful not to keep it.

So the mascot came before the product, and the product turned out to be about exactly what the mascot already did.

— Read the book · Run a node

Elimination beats lookup​

Gnarl never asks "who has this document?" It asks "who can I rule out?"

Every node publishes a manifest: a compact, signed statement of what it can answer. A few kilobytes, whether the node holds fifty thousand documents or fifty million.

{
  "index": "imagery",
  "doc_count": 48219,
  "fields": ["title", "captured_at", "sensor"],
  "summary": {
    "time_range": ["2019-03-01T00:00:00Z", "2026-07-28T00:00:00Z"],
    "bbox": [-124.7, 24.5, -66.9, 49.4],
    "terms": "bloom:8kb:base64…"
  },
  "signature": "ed25519:9c41…2f7e"
}

Given a query, the planner walks its cached manifests and discards every node it can prove cannot contribute:

Whatever survives gets asked.

Why this is safe​

Every summary is conservative by construction. A Bloom filter can say "maybe" when the answer is no, but it can never say "no" when the answer is yes. A bbox is always the enclosing box, never a tighter one. A time range always spans the extremes.

So the failure mode is asking a node that turns out to have nothing — one wasted round trip. The failure mode that would actually hurt, silently skipping a node that had the answer, is structurally impossible.

The cost of being wrong is bounded​

False positives are not free, so queries carry a budget:

{"scope": {"max_nodes": 64, "deadline_ms": 750, "min_coverage": 0.9}}

When more nodes survive elimination than the budget allows, the planner ranks survivors by expected contribution — how strongly the manifest matched, how fast that node has been answering lately, how often it has answered at all — and asks the best ones first.

The response reports exactly what happened, so "we asked 64 of 300 eligible nodes" is a fact the caller gets to see rather than a detail buried in the runtime.

Propagation​

Manifests gossip. A node pushes a new version to a random subset of peers, which forward it on until the network converges — seconds, on networks of hundreds of nodes.

Versions are monotonic and signed. A stale manifest cannot overwrite a fresh one, and a peer cannot publish a manifest on somebody else's behalf.

What this buys​

Adding a node is starting a process. It indexes what it can already read, gossips a manifest, and starts receiving the queries it can answer. Nothing rebalances. No map is updated. Nobody is told to make room.

More detail in Routing and manifests.

That assumption used to be free. It is not free anymore.

The bill for centralization​

Three things changed at once.

Data got heavier than the compute. A petabyte imagery archive costs more to move than to search. Ingestion is no longer a setup step you do once — it is a permanent tax you pay forever, in egress, in duplicate storage, and in the lag between when data lands and when it becomes searchable.

Boundaries became legal, not architectural. Residency rules, partner agreements, and classification levels are not problems you solve with a faster pipeline. Some data will never be allowed into your cluster, no matter how good the cluster is.

The blast radius stopped being acceptable. One coordinator, one shard rebalance at the wrong moment, one config push — and the search surface for the whole organization goes dark together.

What we are building instead​

Gnarl replaces the cluster with a network. Every place your data already lives becomes a node: it indexes locally, publishes a signed summary of what it can answer, and answers queries against its own data.

gnarl node up --data ./corpus
gnarl join --network acme-internal
gnarl search "glacier retreat"

There is no coordinator in that network. The node a query happens to arrive at plans the fan-out for that query, and stops being special the moment it returns results.

The manifest is the whole trick​

The hard part of a network without a coordinator is not executing a query. It is deciding who to ask without a central index of who has what.

Every node publishes a manifest — a few kilobytes describing what it can answer. Time ranges. Bounding boxes. Field names. A Bloom filter over its vocabulary. Never the documents.

Planning is then a process of elimination rather than lookup. The planner does not ask "who has this document?" It asks "who can I prove cannot help?" and asks everyone who is left. Summaries are conservative by construction, so a false positive costs one wasted request and a false negative cannot happen.

Partial answers, honestly reported​

In a network, some nodes will be slow, some will be offline, and some will refuse you. That is not an outage — it is one fewer answer. Every response says so:

{
  "coverage": {"eligible": 41, "answered": 39, "timed_out": 2, "complete": false}
}

We think this is strictly more honest than a cluster that quietly returns results from the shards that happened to be up. You can require completeness, retry the stragglers, or degrade on purpose. What you cannot do is mistake a partial answer for a whole one.

Where this is going​

Gnarl is early. The wire protocol and manifest format are documented and stable, and the client SDKs you build against are open source at github.com/lucenia/gnarl-client. The node itself is a commercial product from Lucenia, with a free tier for running a single peer.

If your data is somewhere it cannot leave, we would like to hear about it.

— Install a node · Read the architecture