AI Research Platform for a Regulated Accounting Profession

At a Glance

The Product

The client is planning a research and workflow platform for accounting and tax practitioners in a heavily regulated market. In the proposed product, practitioners would ask complex tax questions and receive cited answers drawn from authoritative government sources, upload spreadsheets for AI-assisted analysis, and generate client advice letters with embedded citations for export to PDF and Word.

The defining constraint is the profession itself. For an accountant, a fabricated ruling number is not a UX bug, it is a professional indemnity issue. The discovery process set non-negotiable design objectives: 95%+ accuracy on gold-standard queries, zero fabricated citations, all data and LLM inference inside the country's cloud region, a seven-day freshness design target for the knowledge base, and full audit reproducibility for every answer. These are targets for the reference architecture, not observed service levels.

The engagement centered on four objectives:

• Design an agent architecture where unverified output structurally cannot reach the user.
• Specify a RAG layer over regulatory sources with citation-grade metadata and a defined freshness target.
• Propose multi-tenant isolation and an in-region deployment model suitable for regulated firms.
• Define eight reference domain skills in a framework where future skills could be content additions rather than code changes.

The Challenge

The core tension of this project was capability versus defensibility. Generative AI is fluent by default and accurate by engineering, and in this domain, fluency without verification is a lawsuit.

1. Hallucination as a hard failure, not a quality issue. Most RAG systems tolerate occasional wrong answers. Here, a single invented section reference could expose a firm professionally. Verification could not be a scoring layer; it had to be a gate that fails closed.

2. A knowledge base that ages by the week. Tax rulings get superseded. An answer citing last year's ruling as current is wrong even if retrieval worked perfectly. The pipeline needed change detection, version supersession, and a freshness ceiling measured in days.

3. Multi-tenancy in a trust-sensitive market. Every firm brings its own policies and precedents. Firm A's documents must be structurally invisible to Firm B, while both share the same regulatory knowledge, in the same infrastructure.

4. Regulated deployment boundaries. The proposed design would keep processing inside a single national cloud region, use in-region LLM endpoints, retain immutable audit records to meet the stated retention requirement, and document a path from managed services to self-hosted infrastructure for more conservative adopters.

My Approach

I designed the system as a layered architecture where each layer has one responsibility and compliance is enforced at the platform level, not sprinkled through business logic.

A state machine, not a free-form loop. The reference agent is designed as an explicit LangGraph state machine: parse, route, execute, verify, compose. The intended control flow makes each step replayable and individually testable and is designed so no route through the graph can bypass verification.

RAG built for citations, not just retrieval. The reference design calls for a Weaviate instance with namespace isolation: a shared global namespace for rulings, legislation, and guidance, plus one isolated namespace per firm. Retrieval would be hybrid, combining dense semantic search and BM25 with cross-encoder reranking from top-50 to top-10. Each chunk would carry citation-grade metadata such as ruling ID, section path, effective date, supersession links, and status, and only current-status chunks would be eligible to support a primary citation.

A four-tier verification layer. In the proposed flow, generated claims would be extracted atomically, checked against retrieved sources by a local NLI cross-encoder, escalated to an LLM judge for borderline cases, and hard-failed if anything were classified as fabricated. The design objective is to return a verification error rather than an unsupported answer.

Skills as data, not code branches. The reference architecture defines eight domain-skill concepts, covering areas like shareholder loan analysis, trust distributions, fringe benefits, and capital gains events, as filesystem-based Agent Skill packages following the Anthropic specification. Each proposed package would bundle its metadata, retrieval strategy, output schema, and scripts, with progressive loading intended to keep metadata near the roughly 100-token design estimate per skill until invocation. A future ninth skill could then be added without changing the agent graph.

Async by default, scaled with Celery and Dagster. The design assigns advice-letter generation to Celery jobs so firm-customized documents with footnoted citations could be rendered to PDF, Word, and Excel without blocking the API. Dagster is proposed for a daily ingestion pipeline covering browser-automated scraping of the regulator's site, change detection, chunking, embedding, and supersession management. That cadence is intended to support the seven-day freshness design target; it is not a measured guarantee.

Compliance as infrastructure. The proposal places FastAPI services on AWS ECS in the required region, uses PostgreSQL row-level security for operational state, and records each query's retrieved chunks, prompt, model version, and verification result for audit reproducibility. A self-hosted memory option using Mem0, Weaviate, and Neo4j is included so institutional research memory could remain inside the client's boundary.

Engineering Highlights

• Verification-gated LangGraph state machine. Designed to prevent unverified output from reaching a user, with a hard-failure graph node and incident logging.
• Hybrid RAG with citation metadata. Proposed dense plus BM25 retrieval, cross-encoder reranking, and chunk-level ruling IDs, effective dates, and supersession tracking.
• Namespace-per-tenant isolation. A proposed shared regulatory namespace and isolated firm namespaces, merged at query time with sources distinguished in citations.
• Agent Skills framework. Eight reference skill concepts as discoverable filesystem packages with progressive token loading, intended to be extendable without changing the agent graph.
• FastAPI plus Celery latency plan. P50 under 8 seconds and P95 under 20 seconds are latency objectives, not measured performance; slower letter generation and bulk imports would move to async workers.
• Dagster ingestion proposal. A daily pipeline with change detection and version supersession is intended to support the seven-day freshness design target.
• Proposed region-locked AWS deployment. The design would pin compute and inference in-region, apply row-level security, and retain an immutable audit trail subject to validation against statutory requirements.
• Self-hosted memory option. Mem0 with Weaviate and Neo4j could give firms a searchable, relationship-aware record of past research while keeping it inside their environment.

The Impact

The following is expected design value, not delivered or measured impact:

• A defensible answer path. If implemented and validated, the verification gate and audit trail would help a client explain an answer to a professional standards body.
• A route to serving regulated firms. The proposed in-region processing, tenant isolation, and self-hosted option are intended to make more conservative firms addressable.
• Content-led extensibility. The skill-package design is intended to let future skills, firms, and rulings fit existing structures with less engineering work.
• A cost-control strategy. Progressive skill loading, prompt caching, and a small local NLI model are proposed mechanisms for keeping per-query economics within the target model; no realized savings are claimed.

Tech Stack and Deliverables

• Reference platform: LangGraph, LangChain, FastAPI, Claude with proposed in-region endpoints, AWS ECS
• Reference knowledge and data concepts: Weaviate for hybrid, multi-namespace search; PostgreSQL with row-level security; Mem0 with Neo4j for graph memory
• Proposed pipelines and workers: Celery for advice-letter generation and bulk imports; Dagster for daily regulatory ingestion with change detection
• Verification design: Claim extraction, local NLI cross-encoder, LLM judge, hard-fail citation gate
• Design deliverables: Target MVP architecture, agent and RAG specifications, verification design, eight reference skill concepts, multi-tenant and compliance infrastructure design, ingestion design, and cost and scaling model