September 3, 2026 · 7 min read
I built a RAG service for a regulated finance and legal practice and called it adaptive-rag. It isn't. There is no LLM router picking a path. Vector search always runs, web search is added only when a person flips a toggle, and four self-reflection gates with hard-capped loops decide whether the answer is good enough to ship. Here is why that design beat the classic one.
The repo is called adaptive-rag. The first line of its README says it is not adaptive RAG. Both are true, and the gap between them is the most useful design decision in the project.
Classic adaptive RAG puts an LLM at the front door. The model reads the question, decides whether to answer from the vector store or go to the web, and picks one route. It sounds smart. In a regulated finance and legal practice it is a compliance problem waiting to happen: a model that silently decides to leave the knowledge base and read the open web, with nobody able to say afterwards why it did.
So I moved the decision. The router is a person.
Every request to POST /chat carries three things: the question, a thread_id, and a boolean web_search. The vector store is searched on every single request, no exceptions. Web search is additive. It runs only when the toggle is on, and it never replaces the vector results, it sits beside them.
That gives you three properties you cannot get from a model-driven router:
The trade-off is honest: the user has to know when to flip the switch. In practice they know far better than the model does. A practitioner asking about a rule they use every week wants the curated knowledge base. The same practitioner asking about a ruling published last month wants the web. They can tell the difference. The router cannot.
The model does make decisions, just not about where to look. It decides whether what came back is good enough. That is the self-reflection part, and it is the reason the pipeline is a graph rather than a chain.
Twelve nodes. The spine is contextualize (condense the thread history into a standalone question), retrieve_vector (Milvus, HNSW, cosine), rerank_vector (a cross-encoder cuts the candidate pool to the top N), then the first of four gates.
Gate 1, grade_documents. Each retrieved chunk gets one LLM call: is this relevant to the question, yes or no? Anything graded no is dropped. If nothing survives and the toggle is off, the graph does not generate. It routes to rewrite_query, tries retrieval once more with a keyword-richer question, and if that also comes back empty it routes to give_up, which returns a plain "I don't have enough in the knowledge base to answer this reliably" and no fabricated citations.
Gates 2 and 3, reflect. After generate streams an answer, two more calls: is the answer grounded in the context it was given? Does it actually address the question? Not grounded routes to prepare_regen and generates again. Off topic routes back to rewrite_query and retrieves again.
Gate 4 is the rewrite itself. It is a retry with a better question, and it sits inside both of the other loops.
Two loops that can each send the graph backwards is how you get a pipeline that spins forever on a bad question and a small model. The fix is boring and it is the part I am proudest of.
Both loops are hard-capped. MAX_REWRITES and MAX_REGEN both default to 1. The counters are incremented in exactly one node each (rewrite_query and prepare_regen), both routing functions read the counter before they route, and nothing resets a counter mid-turn. contextualize zeroes them once at the start of every request and never again.
I traced every path. With the defaults, the worst case is two extra passes, then finalize. There is no input that makes it loop. That property is worth more than any prompt.
Two details at the output end matter more than they look.
The context handed to generate is source-labelled. Every chunk arrives as [REFERENCE] (a curated practitioner note), [VECTOR] (an ingested official page) or [WEB] (a live fetch), numbered [[1]], [[2]] and so on. The model cites by number. The client receives a sources event before the first token, so it can resolve every [[n]] the moment it appears.
And each citation is a deep link. For vector and web sources the URL carries a text fragment (#:~:text=) built from the cited passage, so clicking it opens the official page scrolled to the exact sentence. In a domain where "show me where it says that" is the whole job, that one feature earned more trust than anything the model said.
Reviewing the code for this post turned up two things I would not have admitted a month ago.
The reranker was blocking the event loop. The cross-encoder is synchronous, CPU-bound torch work, and I was calling it inline from an async node. Under concurrent load, one request's reranking froze every other request's token stream for the duration. The DuckDuckGo call in the same file was already wrapped in asyncio.to_thread. The reranker now is too. The README had a whole section on diagnosing latency with per-node elapsed_ms logs, and the biggest source of it was a missing to_thread.
The rate limiter could be reset by the caller. The Redis key was {client_ip}:{thread_id}, and thread_id is client-supplied. A caller that sends a fresh thread id per request never accumulates a count. The key is now the client alone. While I was there, a Redis blip used to turn into a 500 for every chat request; the limiter now fails open with a warning in the log, because losing the limiter for a minute is better than losing the service.
Both fixes are in the repo with tests.
This is the first of three posts on the system. Next: Skills as Sub-Agents, how one SKILL.md and one skill.yaml parametrize four points of this same graph, run on an isolated thread, and carry their own allowlist.
The regulated-domain thread continues from the verification layer post. That one was about catching fabricated citations after generation. This one is about retrieving well enough that there is less to catch.
Written by
Vishvdeep Dashadiya
Lead AI Engineer. Agentic AI, real-time ML systems, and cloud-native infrastructure.
Keep reading
September 20, 2026 · 6 min read
Jev is a model that never generates text — it answers typed questions with calibrated probabilities. Where that fits in a LangGraph agent, with code, and where I would not use it.
September 5, 2026 · 8 min read
Every run of the pipeline becomes a nested Langfuse trace, and a judge model scores it with DeepEval. Neither adds a millisecond to the response, because scoring starts after the client already has the answer. The wiring: a per-request callback, a fire-and-forget task that cannot fail the request, a judge decoupled from the graph LLM, and an offline golden set for the metrics live traffic cannot compute.
September 4, 2026 · 7 min read
A skill is a folder with two files. SKILL.md becomes the system prompt, skill.yaml declares which sources the run may read, and the shared LangGraph pipeline runs unchanged on an isolated thread. Slash-invocable, like a command. Here is how a domain persona parametrizes four points of one graph without forking it, and the allowlist bug a code review found along the way.
Tell me about it. I reply within one working day with a first take and no sales pitch.
