Project
Adaptive RAG
Toggle-Driven Hybrid Retrieval With Skills
A self-hosted LangGraph and FastAPI RAG service for a regulated finance and legal practice. Vector search always runs; allowlisted web search is added only when the request toggles it on, never by an LLM router. Four self-reflection gates with hard-capped retry loops decide whether an answer ships. Slash-invocable skills run as isolated sub-agents over the same graph, and opt-in Langfuse tracing plus async DeepEval scoring observe every run without adding latency.
- LangGraph
- FastAPI
- Milvus
- Redis
- Ollama
- bge-reranker
- Langfuse
- DeepEval
- Python
Key Features
Human-Chosen Route
The vector store is searched on every request. Web search is additive and switched on by a per-request toggle, so the route is a field in the trace rather than a model decision nobody can audit.
Four Self-Reflection Gates
Per-document relevance grading, a groundedness check, an answer-quality check and a query-rewrite retry. Both retry loops are hard-capped, so the graph always terminates or gives up honestly.
Skills as Sub-Agents
A SKILL.md and a skill.yaml parametrize four points of the shared graph: system prompt, source filter, web allowlist and web mode. Each run is isolated on a derived thread.
Trace and Score Without Latency
A per-request Langfuse callback records every node and LLM call. DeepEval GEval scoring starts after the stream closes, on a judge model decoupled from the graph LLM.
Architecture
- FastAPI endpoint streams thread, status, sources, token and done events over SSE
- LangGraph pipeline with SQLite checkpointing per thread; skill runs on thread_id::skill
- Milvus (HNSW, cosine) with per-skill metadata filters; content-hash refresh on ingest
- Allowlist enforced three times: site: filter, pre-fetch host check, post-redirect host check
- In-process cross-encoder reranker runs on a worker thread off the event loop
- Citations deep-link to the exact passage with URL text fragments
- Opt-in Langfuse v2 profile; fire-and-forget DeepEval scoring that cannot fail the request
Diagrams
Request and skill routing at the top, the skill-parametrized LangGraph pipeline on the left, self-hosted systems and ingestion on the right, and the opt-in observability profile in the corner.
The compiled graph exported from the live code: retrieve, rerank and grade, then a router that adds web search, builds context, rewrites once or gives up; generate, reflect, and a second router that regenerates once or finalizes.
API Usage
def route_after_grade(state: GraphState) -> str:
settings = get_settings()
if state.get("web_mode") == "fallback":
# Skill runs: vector-first. Web is consulted only when the
# vector docs can't answer and the human toggle is on.
if state.get("relevant"):
return "build_context"
if state.get("web_search") and not state.get("web_searched"):
return "web_search"
if state.get("rewrite_count", 0) < settings.max_rewrites:
return "rewrite_query" # hard cap, default 1
return "give_up" # honest "not in the knowledge base"
# Base "alongside": toggle on means web runs next to vector every time.
if state.get("web_search"):
return "web_search"
if state.get("relevant"):
return "build_context"
if state.get("rewrite_count", 0) < settings.max_rewrites:
return "rewrite_query"
return "give_up"Impact
Practitioners get answers grounded in an allowlisted knowledge base, with every route decision, retrieved chunk and quality score recorded on a single trace, and citations that open the source page scrolled to the cited sentence.
4
Self-Reflection Gates
3x
Allowlist Enforcement
0 ms
Added Eval Latency
Want something like this built for you?
Tell me about it. I reply within one working day with a first take and no sales pitch.