Vectorless RAG: Do You Still Need a Vector Database in 2026?

For two years, the answer to «how do I make a chatbot answer from my own data?» was always the same: chop the data into chunks, turn each chunk into numbers, store the numbers in a vector database. Then, in mid-2026, a wave of posts declared that step dead. «Vectorless RAG.» «You don't need a vector database anymore.» «RAG is dead.»

We build AI chatbots for a living, so we read the actual papers, ran the numbers, and looked past the headlines —the same way we did with PixelRAG. This is the honest version for someone deciding what to build: what vectorless RAG actually is, how a chatbot finds information in the first place, what a vector database really does, and the practical question underneath all the noise — do you need one? With the real figures, not the viral ones.

What is vectorless RAG?

Vectorless RAG is retrieval without a vector database. Instead of converting your documents into numerical vectors and searching by mathematical similarity, the system uses the document's own structure — or plain keyword search — and lets a language model reason about which part is actually relevant. RAG stands for Retrieval-Augmented Generation: the technique most AI chatbots use to answer from your data instead of making things up. «Vectorless» changes the retrieval half of that.

The poster child is PageIndex, from VectifyAI. It builds a hierarchical «table of contents» tree from a long document, then, when a question comes in, an LLM walks that tree the way a person flips through a report — «this is a financial filing, the answer is probably in the liquidity section, page 40» — and pulls the right section. No chunking, no embeddings, no vector store. Their tagline captures the whole pitch: similarity is not relevance.

There's a second flavour that's even simpler: give an AI agent a plain keyword search tool (the kind of full-text search a database has had for decades) and let it search, read, refine, and search again in a loop. That's how coding agents like Claude Code and Cursor navigate a codebase — they grep, they don't embed. A recent paper by researchers at Amazon put a number on it (more on that below).

How an AI chatbot actually finds information (RAG in 60 seconds)

To judge whether removing the vector database is a good idea, you need to know what it was doing there. Here is the standard RAG pipeline that powers the vast majority of AI chatbots today, ours included:

1. Chunk. Your documents — a PDF, a web page, a product catalog — get split into small passages of text.
2. Embed. Each passage is fed to an embedding model that turns it into a list of numbers (a vector) capturing its meaning. Two passages about the same idea end up with similar numbers, even if they use different words.
3. Store. Those vectors go into a vector database built to find nearest neighbours fast.
4. Retrieve. When a customer asks something, the question is embedded too, and the database returns the passages whose vectors sit closest to it.
5. Generate. Those passages get handed to the language model, which writes the answer grounded in them.

The vector database lives in step 3 and 4. It exists to answer one question quickly: which of my thousands of passages are most similar in meaning to this query? Vectorless approaches argue that «most similar in meaning» is often the wrong question — and that a language model, given the document's structure or a search tool, can find the genuinely relevant part more reliably.

What a vector database is (and what embeddings are)

An embedding is a way of turning text into a point in space. The embedding model reads «waterproof jacket» and outputs a long list of numbers — say 1,536 of them — that pin that phrase to a specific coordinate. «Rain coat» lands nearby because it means something similar; «diesel engine» lands far away. The whole trick is that closeness in this space ≈ closeness in meaning.

A vector database (Pinecone, Weaviate, Qdrant, pgvector on top of PostgreSQL, and others) is storage optimised to hold millions of those points and, given a new one, return the nearest neighbours in milliseconds. That's it. It's a very good tool for one specific job: fuzzy, meaning-based matching over a big pile of text where you don't know the exact words the user will type.

It's also where a lot of the cost, complexity and failure modes of RAG live: chunk boundaries that cut a table in half, an index to keep in sync, and the core assumption that similarity equals relevance. That's where the vectorless argument starts. In a lot of real cases, going for the closest match doesn't get you the right answer, so building and maintaining all that infrastructure isn't worth it.

Semantic search vs keyword search

This is the fault line under the whole debate, so it's worth a plain example. Same knowledge base, same question, two ways of finding the answer:

Neither is universally better. Semantic search shines when the user's words won't match your document's words. Keyword and structured search shine when the answer hinges on an exact value — a price, a size, a reference, a date. The mistake the last two years made was treating vectors as the default for everything, including questions where a plain filter would have been faster, cheaper and more accurate.

Is RAG dead? What the debate actually says

Short answer: no. What's being questioned isn't «grounding a model in your data» — that's more relevant than ever. What's being questioned is one specific implementation: chunk everything, embed it, and search by vector similarity. Three threads fed the «RAG is dead» wave, and each says something narrower than the headline:

• Bigger context windows. Models now take huge inputs, so for a small knowledge base you can sometimes just paste everything in and skip retrieval entirely. This breaks down at scale and gets expensive — and it collides with the next point.
• Context rot. Chroma's research showed model accuracy degrades as the input grows, well before the window is full. So stuffing everything in isn't free — the model gets worse the more you add. If anything, context rot is an argument for better retrieval, not for dropping it.
• Agentic search. Give a capable model a search tool and let it look things up in a loop, and it often beats a one-shot vector lookup. This is the real substance behind vectorless RAG.

So the honest framing is not «RAG is dead» but «the reflexive vector-database-for-everything era is ending». Retrieval is alive; the monoculture around one method is what's fading.

How vectorless RAG works under the hood

Take PageIndex as the concrete example. Instead of chunking and embedding, it does this:

1. Build a tree. It generates a hierarchical, table-of-contents-style index of the document — sections, subsections, page anchors — that mirrors how the document is actually organised.
2. Reason over the tree. When a question arrives, an LLM does tree search: it looks at the structure and decides which branch is likely to hold the answer, navigating like a human skimming a report, rather than matching vectors.
3. Retrieve the relevant node. It pulls the section it reasoned its way to, and the model answers from that.

The keyword-agent flavour skips even the tree: the agent issues full-text searches, reads results, and refines its query until it has what it needs — the same loop a person uses with a search box.

Do you need a vector database? A decision guide

Forget the ideology. Run through these and count where you land:

• Mostly yes: a vector database still earns its place. Semantic search over a big, messy corpus is exactly what it's good at. Keep it.
• Mixed: you probably want hybrid — keyword/structured search for exact stuff, vectors for the fuzzy stuff. Most serious systems already blend both.
• Mostly no: if your data has clear structure (well-organised documents) or your questions hinge on exact values (catalogs, records, filters), vectorless — a reasoning tree or plain structured search — may be simpler, cheaper and more accurate.

Vector RAG vs vectorless RAG: an honest table

The row that matters most is the last one: vector RAG is a known quantity; vectorless is promising but young, and most of its winning numbers come from the people selling it. That's not a reason to ignore it — it's a reason to test it against your own data rather than a headline.

What this means for your business chatbot

Here's the part the debate usually skips: most business chatbots never needed pure vector search in the first place. A store chatbot answering «waterproof jackets under €80 in size L» is not a similarity problem — it's a filter. The right answer is structured search over your catalog (price, size, stock), not the nearest vector to the sentence. We wrote about exactly this in our guide on why your chatbot can't find products.

At Bravos AI we already run a hybrid: structured, database-style search for catalogs and exact-value questions, and semantic retrieval for free-form text like FAQs, policies and service descriptions. The «vectorless» conversation didn't surprise us because the lesson underneath it — match the retrieval method to the question, don't force vectors on everything — is how a good system should have been built all along. If your chatbot mostly answers questions about a catalog or structured records, the vector database was never the important part.

The verdict, in one line

Vectorless RAG is a real, useful correction to two years of over-using vector databases — not the death of RAG. For structured documents and exact-value questions it can be simpler, cheaper and more accurate; for large, fuzzy, meaning-based corpora, a vector database still earns its keep. The right question was never «vectors or no vectors» — it's «which retrieval fits this question». If a vendor tells you one method wins every time, they're selling, not engineering.

Frequently asked questions

What is vectorless RAG in simple terms?

It's retrieval-augmented generation without a vector database. Instead of turning your documents into numerical embeddings and searching by similarity, the system uses the document's structure (a reasoning tree, like PageIndex) or plain keyword search, and lets a language model reason about which part is actually relevant. The tagline is «similarity is not relevance».

Do I still need a vector database for RAG in 2026?

It depends on your questions. For large, unstructured corpora where users ask fuzzy, meaning-based questions, yes — vector search is what it's good at. For well-structured documents or questions that hinge on exact values (catalogs, records, filters), you may not, and a vectorless or hybrid approach can be simpler and cheaper. Many production systems use both.

Is RAG dead?

No. Grounding a model in your own data is more relevant than ever. What's fading is the reflex of chunking and embedding everything into a vector database by default. Retrieval is alive; the one-size-fits-all method is what's being questioned.

What is PageIndex?

PageIndex (by VectifyAI) is a vectorless, reasoning-based RAG system. It builds a hierarchical table-of-contents tree from a document and uses an LLM to navigate it, with no embeddings, chunking or vector database. Its authors report 98.7% accuracy on the FinanceBench benchmark — a vendor-reported figure, so worth verifying against your own data.

Vectorless RAG vs vector RAG — which is better?

Neither wins outright. Vector RAG is mature, fast and strong at fuzzy semantic matching over big corpora. Vectorless RAG is newer and strong at structured documents and exact-value questions, with simpler infrastructure but higher per-query reasoning cost. The best answer for most businesses is a hybrid that routes each question to the method that fits it.

Sources

• PageIndex (VectifyAI): github.com/VectifyAI/PageIndex — vectorless, reasoning-based RAG; 98.7% on FinanceBench (vendor-reported).
• «Keyword search is all you need»: Subramanian et al., arXiv:2602.23368 — agentic keyword search reaches over 90% of traditional RAG performance without vector databases.
• Context Rot (Chroma): research.trychroma.com/context-rot — model performance degrades as input context grows.