September 4, 2026 · 7 min read
A skill is a folder with two files. SKILL.md becomes the system prompt, skill.yaml declares which sources the run may read, and the shared LangGraph pipeline runs unchanged on an isolated thread. Slash-invocable, like a command. Here is how a domain persona parametrizes four points of one graph without forking it, and the allowlist bug a code review found along the way.
The first version of the system had one pipeline and one system prompt. Then the practice asked for a second persona: not a general researcher but a retirement-planning strategist, with a different voice, a narrower set of sources, and a different idea of when to reach for the web.
The tempting move is to copy the graph. Two pipelines, two prompts, two sets of bugs. I did not want two of anything. What I wanted was for the same graph to behave like a different specialist depending on who invoked it.
That is what a skill is. Same graph, different contract.
skills/regulatory-research/
SKILL.md # YAML frontmatter + markdown body
skill.yaml # sources, allowlist, web mode, ingestion seeds
references/ # optional curated markdown
SKILL.md has a frontmatter block with a name and a description (the description powers the / menu in the UI), and a markdown body. The body is the system prompt for a run under that skill. Everything about voice, output format, source hierarchy and how to handle uncertainty lives there, in prose a domain expert can edit without touching Python.
skill.yaml is the contract:
name: regulatory-research
web_mode: fallback
vector_sources: [regulator.example, legislation.example, caselaw.example]
web_allowlist: [regulator.example, legislation.example, caselaw.example,
professional-body-one.example, professional-body-two.example]
ingest:
references: true
topics:
- act_section: "Related-party loan rules"
topic: "loans from a private company to its owners"
query: "private company loan to shareholder deemed distribution complying loan"
The references/ bundle is a set of paraphrased practitioner notes. It is ingested into the vector store tagged skill=regulatory-research and source_type=reference, so retrieval ranks curated notes alongside crawled official pages, and the citation can still deep-link to the canonical URL declared in the note's frontmatter.
A SkillRegistry scans skills/*/SKILL.md once at startup. Adding a persona is: create the folder, restart, ingest. No code.
Two ways, and they resolve to the same thing:
{ "question": "/regulatory-research how do the related-party loan rules work?", "thread_id": "t-1", "web_search": false }
{ "question": "how do the related-party loan rules work?", "thread_id": "t-1", "skill": "regulatory-research" }
A leading /slug token is parsed off the question, exactly like a slash command. The explicit skill field wins if both are present. An unknown slug is a 400, not a silent fallback to the generic pipeline, because a practitioner who typed /retirement-planning and got a generic answer would not notice, and that is worse than an error.
The graph does not know skills exist. Its nodes look up state["skill"] in the registry and read four values:
| Node | What the skill supplies |
|---|---|
contextualize | web_mode, which decides how the router treats the web toggle |
retrieve_vector | a Milvus filter built from vector_sources, so the skill only sees its own references plus official pages from its declared hosts |
web_search | web_allowlist, the only hosts the fallback may fetch |
generate | the SKILL.md body as the system prompt |
Everything else, the reranker, the four gates, the loop caps, the citation numbering, is identical for every skill. Fix a bug once and every persona gets the fix.
The filter in retrieve_vector deserves a sentence. It is a boolean expression over the collection's metadata: (source_type == "reference" and skill == "<slug>") or (source_type == "vector" and domain in [...]). A skill can therefore be regulator-only. The retirement-planning skill declares the same three official hosts but a different professional body, so its web fallback reads a specialist industry association the research skill never sees.
thread_id::skillA skill run happens on a derived thread. If the client thread is t-1, the research skill's memory lives at t-1::regulatory-research in the SQLite checkpointer, and the plain chat thread at t-1 never sees it.
This matters because contextualize condenses prior turns into the current question. Without isolation, a follow-up in the generic thread would be rewritten against a specialist's conversation, and a specialist's follow-up against generic chat. Each persona keeps its own history and the client keeps one id.
The response's done event carries the skill name back, so the frontend can label the thread.
The base /chat path runs web search alongside vector whenever the toggle is on. Skills flip that to fallback: web runs only when the vector documents cannot answer (nothing survives the relevance gate) and the toggle is on. The knowledge base is primary and the web is a safety net, which is exactly how the practitioner thinks about it.
A one-shot guard (web_searched) means the fallback fires at most once per turn. After a rewrite it will not fire again, even though the toggle is still on. That was deliberate, and the graph's termination proof depends on it.
The allowlist is the safety property of the whole system. The glossary in the repo said it was "enforced twice", and for months I believed it.
Check 1, on the query. The search runs once per allowed domain with a site: filter appended, so the search engine only returns candidates from those hosts.
Check 2, before the fetch. Every returned URL's host is checked against the allowlist: an exact match or a subdomain. Search-engine noise and lookalike domains are dropped here.
Check 3, after the fetch. This one did not exist. The HTTP client followed redirects, and the host was verified on the URL we asked for, never on the URL we got. An allowlisted page that returned a 302 to an arbitrary host would have had its content fetched, extracted, and quoted back to a practitioner with an official-looking citation.
The fix is a few lines: check the host of response.url after the redirect chain, drop and log if it is off-list, and cite the final URL rather than the pre-redirect one so the citation points where the content actually came from. It came with a test that mocks a 302 to an off-list host and asserts the document is dropped, a test that a same-host redirect is kept with the final URL recorded, and a test for lookalike domains.
The docstring and the glossary now say "three times". The gap between what documentation claims and what code does is where these bugs live.
When the second skill arrived, the whole change was two files, a restart and one ingestion command:
python -m app.ingestion.run_ingest --skill retirement-planning
curl -s localhost:8000/api/v1/skills # both listed
Ingestion for a skill pulls its references/*.md (tagged reference) and runs each ingest.topics seed through the same allowlisted search-and-fetch path the live fallback uses. So what the vector store holds is exactly what a live web fallback would have fetched. There is one code path for reading the web, and the allowlist sits in it.
SKILL.md for the persona, skill.yaml for the contract, an optional references bundle. The registry loads it at startup.thread_id::skill keeps each persona's memory apart while the client holds one id.Previous: It's Not Adaptive RAG. Next: Tracing and Scoring a RAG Pipeline Without Slowing It Down, how every run becomes a nested trace and gets scored by a judge model after the client already has its answer.
Written by
Vishvdeep Dashadiya
Lead AI Engineer. Agentic AI, real-time ML systems, and cloud-native infrastructure.
Keep reading
September 20, 2026 · 6 min read
Jev is a model that never generates text — it answers typed questions with calibrated probabilities. Where that fits in a LangGraph agent, with code, and where I would not use it.
September 5, 2026 · 8 min read
Every run of the pipeline becomes a nested Langfuse trace, and a judge model scores it with DeepEval. Neither adds a millisecond to the response, because scoring starts after the client already has the answer. The wiring: a per-request callback, a fire-and-forget task that cannot fail the request, a judge decoupled from the graph LLM, and an offline golden set for the metrics live traffic cannot compute.
September 3, 2026 · 7 min read
I built a RAG service for a regulated finance and legal practice and called it adaptive-rag. It isn't. There is no LLM router picking a path. Vector search always runs, web search is added only when a person flips a toggle, and four self-reflection gates with hard-capped loops decide whether the answer is good enough to ship. Here is why that design beat the classic one.
Tell me about it. I reply within one working day with a first take and no sales pitch.
