hypREspace: Cutting Token Costs 90% While Raising Tool-Calling Accuracy to 95%

At a Glance

The Product

hypREspace is an AI-driven transaction management platform for corporate real estate, trusted by firms including Cushman & Wakefield, which selected it as their transaction management technology provider in 2025. Its agentic AI features run in production, on real user traffic, every day.

Production is where agent economics get honest. The agent worked, but every query carried a large static prompt: full workflow instructions, tool guidance, and formatting rules, whether the query needed them or not. Token costs scaled with traffic, and tool-calling mistakes, the wrong tool, wrong arguments, or the wrong order, meant wasted retries on top. The mandate was direct: make the agent cheaper and more accurate at the same time, and prove both with numbers.

The engagement centered on four objectives:

• Cut token spend by making prompt content dynamic instead of static.
• Raise tool-calling accuracy: right tool, right arguments, right order.
• Build an evaluation harness so both improvements are measured, not assumed.
• Fine-tune a model on the agent's own successful behavior to lock the gains in.

The Challenge

The core tension of this project was capability versus cost. Everything you put in a prompt makes the agent theoretically more capable and definitely more expensive, and past a point, bigger prompts make tool calling worse, not better.

1. The static prompt tax. Every query paid for the full instruction set, even when it used a fraction of it. A simple lookup carried the same prompt weight as a complex multi-step workflow. At production volume, that overhead compounds into real money.

2. Accuracy you cannot see is accuracy you cannot fix. Tool-calling failures were anecdotal: a wrong argument here, a skipped step there. Without a ground-truth benchmark there was no way to know the real accuracy, which meant no way to know if any change helped.

3. Prompt bloat degrades tool selection. Long prompts bury the instructions that matter for the current query. The agent had to find the relevant workflow guidance inside everything else, and sometimes it did not.

4. Improvements had to hold at lower cost. The end goal was not just a tuned prompt on an expensive model. It was locking reliable behavior into the system so quality and cost improve together, including through fine-tuning.

My Approach

I ran this as an evaluation-first engagement. Nothing shipped without a number attached.

A gold evaluation dataset before any changes. I built a gold-standard eval set of real query patterns, each annotated with the correct tool calls: which tool, with which arguments, in which order. Accuracy became a score, not an impression. Token usage was instrumented the same way, with a token counter measuring every query before and after each change.

Dynamic prompt injection through the custom MCP layer. Instead of one giant static prompt, I restructured instructions into portions served by the MCP layer. The agent calls a tool to fetch exactly the workflow or prompt section the current query needs, when it needs it. A simple query pulls a small prompt. A complex workflow pulls its full instructions. The prompt became pay-per-use.

Prompt tuning against the eval set. With the harness in place, prompt variants were iterated and scored against the gold dataset. Changes that raised tool-calling accuracy stayed, changes that did not were discarded, and every decision had a before-and-after number.

Fine-tuning to lock it in. The final layer was training the behavior into the model itself. I curated roughly 10,000 verified golden trajectories from production logs, filtered for correct tool calls and completed workflows, and fine-tuned Amazon Nova with SFT and LoRA using Unsloth. Verification ran on two tracks: programmatic verifiers checking tool names and arguments, and LLM-as-judge evaluation on a held-out set. A model trained on the agent's exact tool-call format emits valid calls consistently, without carrying an 80,000-token instruction manual into every query.

Engineering Highlights

• Gold eval dataset as the source of truth. Tool-calling accuracy scored on right tool, right arguments, and right call order against annotated ground truth.
• Token accounting on every query. Before-and-after token counts and costs measured with a token counter, so the savings figure is arithmetic, not a guess.
• MCP-served prompt portions. Workflow instructions decomposed into fetchable sections; the agent retrieves only what the query requires.
• Prompt tuner loop. Systematic prompt iteration scored against the eval set instead of tweaked by feel.
• ~10K golden trajectories. The SFT dataset came from the agent's own verified production successes, filtered for correct tool calls and completed workflows, so the model learned the system's real behavior, not synthetic approximations.
• Amazon Nova fine-tuned with Unsloth. SFT with LoRA adapters, keeping training fast and affordable while staying inside the AWS ecosystem the client already runs on.
• Two-track verification. Programmatic verifiers checked tool names and arguments; LLM-as-judge evaluation on a held-out set caught the failures rule-based checks miss.
• Compounding gains. Injection cut the cost per query, tuning raised accuracy per call, and fine-tuning locked both into the weights.

Results

Tech Stack and Deliverables

• Platform: LangGraph, LangChain, custom MCP layer, FastAPI
• Fine-tuning: Amazon Nova, SFT with LoRA via Unsloth, ~10K curated golden trajectories from production logs
• Evaluation: Gold-standard tool-calling eval dataset, token counting instrumentation
• Deliverables: Evaluation harness and gold dataset, dynamic prompt injection architecture, tuned production prompts, trajectory-mined SFT dataset, fine-tuned model and serving setup, before-and-after cost and accuracy report