What PaveDB adds to vector search

About a year ago, I built PatchVec in two days for a very specific problem: get a file into search without first assembling a chain of separate components. It was a small vector-search service, and that was enough for the first version. The work that followed made the useful question larger. Once a result arrives, can the application inspect what it matched, trace the source context, revisit the query, and see what happened in operation? PaveDB is the result of treating those needs as part of the retrieval path.

The practical advantage begins with a shorter route from file to search. A retrieval application can ingest a PDF, CSV, or TXT file; PaveDB chunks, embeds, and indexes it, then returns search results from the same path. That does not make every retrieval problem simple, but it avoids making the application stitch together one component for ingestion, another for vectors, and a third place to reconstruct what a result meant. Re-ingesting the same document identifier swaps that document’s content in place, without a restart.

In an embedded Python application, the compact form looks like this:

from pavesdk.client import connect

books = connect("./data").create_collection("books")
books.ingest("manual.pdf", docid="operations-manual")
hit = books.search("rotation procedure", k=3)[0]
query = books.queries(limit=1)[0]
record = books.get_query(query["query_id"])
again = books.replay(query["query_id"])

The first three operations are familiar: turn a file into searchable chunks, then ask for a result. The difference shows in the objects left behind. A hit includes a matched snippet, its document identity, and location metadata. The application can inspect the stored raw text for that chunk later. This gives a reader or operator source context behind a hit instead of only a score and an identifier.

That boundary is intentional. PaveDB’s provenance is text-backed: it exposes the stored chunk text and associated document/location metadata. It is useful evidence for the text that was indexed and matched, but it is not a byte-for-byte preservation system for the original PDF or CSV. A workflow that needs archival preservation still needs to keep the source file. A workflow that needs to explain why a retrieval result appeared has a direct path to the indexed context.

The query record offers a second kind of context. For text searches, PaveDB retains the query, parameters, ordered result IDs, timing, request context, and scope. A saved search can be retrieved later, and replay runs that request again while linking the new record to the earlier one. This makes a prior search available for inspection and comparison without turning replay into a claim that the system has frozen time.

It has not. Replay runs against the current collection, so changes to the corpus, index, embedder, or collection state can produce a different result. That difference can be useful evidence, but it is not a historical snapshot. Raw vectors get their own path. I added vector-only collections, so an application can bring its own embeddings, then ingest and search them through the same client, with dimensions and metadata validated across restarts. The narrower limit is replay: the text-query record does not retain a vector, so a raw-vector search is inspected through that vector path rather than replayed through this one.

The collection in the example also answers a question that is often left to application conventions: which corpus does this result belong to? PaveDB scopes ingest, search, inspection, and query records by tenant and collection. For a local tool, the boundary might simply be books. In a shared service, a tenant and collection let each application name its own corpus rather than reconstructing boundaries after data and queries have mixed. I made that boundary enforced rather than cosmetic: each tenant authenticates with its own token, tenant and collection names are validated before they become storage paths, and per-tenant limits are applied at the store, so a shared deployment keeps corpora apart instead of trusting callers to stay in their lane. Inspection was native from day one, not something bolted on later. The benefit is not an elaborate account model; it is a clear, inspectable retrieval home for each result even when several applications share one instance.

The path can remain the same when access or embedding choices differ. The Python client works with an embedded local instance or a remote HTTP server; the CLI and documented HTTP/OpenAPI surface cover the same basic work from other operating contexts. An Elixir client is also available, with TypeScript just around the corner. Embedding is a collection-level choice, with local and hosted backends available. An application can therefore keep its focus on documents and queries rather than build a second pipeline merely to prepare vectors for storage.

Finally, a retrieval path needs to be visible when it is running, not only when a notebook example succeeds. PaveDB produces logs for searches and ingest, connects requests to responses and log entries with request IDs, and provides liveness, readiness, and metrics endpoints. Those surfaces do not make every failure self-explanatory. They do provide a way to tell whether an instance is available and what a request did before guessing.

PaveDB is still a vector-search database, not a claim that nearest-neighbor search is unimportant. It adds the surrounding path that makes file-to-search work less stitched together: inspectable source context, saved queries that can be compared later, named tenant and collection boundaries, flexible access and embedding choices, and operational signals. It is approaching its first 1.0 preview now, on a compatibility baseline I hold deliberately: released schema changes migrate stored data forward instead of asking an operator to re-ingest, and a newer instance refuses to quietly downgrade the data it inherited. That is the part I care about most, because retrieval that cannot account for itself is a liability the moment its output feeds a decision. PatchVec was the short origin. PaveDB is the retrieval path around it.

For the current source code and product documentation, visit pavedb.org.