RFC-2026-014: Hybrid Vector Search Pipeline

Replacing standalone Qdrant with FalkorDB + Qdrant tiered retrieval

The traditional way is to put off all annoying decisions until the last possible moment.

Summary

This RFC proposes replacing the current standalone Qdrant vector index with a tiered hybrid pipeline that uses FalkorDB for entity-relation traversal and Qdrant only for dense semantic search. The change reduces P95 query latency from 1.4s to a target of 0.4s, eliminates the 18% cross-shard hallucination rate observed in the current system, and brings the pipeline closer to the cognee architecture spec the team aligned on in Q1.

This is Thariq’s “Specs, Planning & Exploration” use case rendered through stoichos: the same content you’d write as a markdown plan, typeset.

NoteStatus

Draft → review → ratification. Two open questions remain (§5.2, §5.4). All other sections settled per architecture review on 2026-04-22.

Problem

The current ingestion pipeline embeds every document chunk into Qdrant as an isolated dense vector. That works for similarity but loses the entity-relation graph implicit in the source corpus. H₂O-pure semantic similarity is the wrong query model for documents that describe a system: relationships matter as much as meaning.

Three observed failures:

1. Schema-constrained extraction fails at scale. Models below 32B parameters fail constrained JSON extraction at 90% hallucination rates per the community benchmark. This is not specific to our corpus — it’s a model-size threshold.
2. Query latency spikes on long-context retrieval. P95 currently sits at 1.4s with the standalone Qdrant index when retrieving across 5000+ document chunks. The [@SmithEtAl2025] paper attributes this to dense-index thrashing under high cardinality.
3. Cross-shard relation queries are impossible. Asking “what does this entity depend on” requires graph traversal. Vector similarity gives nearest neighbors, not transitive closures.

Proposed architecture

The two-tier model is canonical in modern RAG architectures. See Vector Search Engines for AI Applications (Manning, 2025) and the cognee project’s design notes.

flowchart LR
    Source[Document Corpus] --> Ingest[cognify pipeline]
    Ingest --> Graph[FalkorDB<br/>entity-relation triples]
    Ingest --> Vector[Qdrant<br/>dense embeddings]
    Query[User query] --> Plan[Query planner]
    Plan -->|graph traversal| Graph
    Plan -->|semantic similarity| Vector
    Graph --> Merge[Result merger]
    Vector --> Merge
    Merge --> Response[Ranked response]

The query planner decides per-query whether to dispatch graph traversal, semantic similarity, or both, and the result merger reranks based on graph centrality + semantic distance.

Code change

The retrieval module’s main entry point changes from a single Qdrant call to a tiered dispatch:

@@ -42,8 +42,18 @@ class Retriever:     async def retrieve(self, query: str, k: int = 10) -> list[Doc]:-        embedding = await self.embed(query)-        results = await self.qdrant.search(embedding, top_k=k)-        return [self.hydrate(r) for r in results]+        plan = await self.planner.plan(query)+        results = []+        if plan.needs_graph:+            results.extend(await self.graph.traverse(plan.entities, depth=plan.depth))+        if plan.needs_vector:+            embedding = await self.embed(query)+            results.extend(await self.qdrant.search(embedding, top_k=k * 2))+        merged = self.merger.merge(results, plan)+        return [self.hydrate(r) for r in merged[:k]]

WarningMigration window

The cutover requires re-ingesting the entire corpus through cognify. Estimated wall-clock at current ingestion rate (140MB / 35min): 18 hours for the 4.2TB primary corpus. Plan a maintenance window.

Cost projection

Component	Current	Proposed	Δ
Qdrant cluster	$480/mo	$240/mo	−50%
FalkorDB instance	—	$180/mo	new
Ingestion compute	$60/mo	$110/mo	+83%
Total monthly	$540	$530	−2%

Open questions

ImportantQ1: How do we handle entity disambiguation?

Two distinct entities sharing a label (e.g. Apple the company vs. the fruit) currently collapse in Qdrant via embedding similarity. FalkorDB stores them as distinct nodes if cognify extracts them correctly, but cognify’s NER step doesn’t disambiguate. Owner: @christos. Decision needed by: 2026-05-20.

ImportantQ2: Cold-start retrieval for new documents

A document just ingested has graph edges but no extracted-yet relations until the next cognify pass. During that gap, the graph tier returns nothing for queries hitting the new doc. Options: (a) accept stale-by-one-cycle; (b) run cognify per-document on ingest; (c) eager-extract a minimal relation set inline.

Acceptance criteria

TipDefinition of done

• P95 query latency < 0.5s on the production corpus
• Cross-shard hallucination rate < 5%
• Cognify ingestion completes within tolerable batch window (8h max)
• Migration playbook documented; rollback path tested

The two-tier query model is not optional. Without graph traversal, “what depends on this” is unanswerable. Without vector similarity, “what is this similar to” is.

References

• [@SmithEtAl2025] Smith, R. et al. Vector Search at Scale. ACM Computing Surveys, 2025.
• cognee architecture spec, retrieved 2026-04-15.
• Internal: docs/architecture/retrieval-v2.md