Odeon: Self-Improving Voice Agent Optimization Platform
AI products · Creator, sole engineer · shipped
Prompt tuning for conversational agents is guesswork. You tweak, you hope, you ship - with no evidence the new version is better and no difficult traffic to test it against.
Watch: https://youtu.be/GFdSe4-c_xQ
How it works
Runs cycles until the averages meet every threshold; SQLite keeps each run and every prompt version.
- Personas → Simulation
- Simulation → LLM judge
- LLM judge → All pass?
- All pass? → Optimizer
- Optimizer → Agent prompt
- Agent prompt → Simulation
The detail
Odeon replaces that with a measurable loop. It generates adversarial user personas - the lawyer, the crying student, the stubborn negotiator - and runs full simulated conversations against your agent, so it faces genuinely hard traffic rather than cooperative test scripts.
Every conversation is scored on numerical criteria, and the agent must pass all of them to clear a scenario. Allowing a single-metric win is how prompt tuning produces agents that are empathetic and useless.
When the agent fails, an optimizer rewrites the system prompt automatically - and renders the change as a git-style red and green diff, because an automated rewrite you cannot inspect is not an improvement you can ship.
Every run is streamed live over a websocket and archived for replay and comparison.
Python, FastAPI, WebSockets, LangChain, Groq, React, SQLite.
What I did
- Built the full platform - persona generator, simulation runner, scoring engine, optimizer and frontend
- Designed adversarial persona generation so agents face genuinely hard traffic
- Implemented multi-criteria scoring where all thresholds must pass
- Built the meta-agent that rewrites the system prompt on failure
- Built the git-style diff view so every automated rewrite is inspectable
- Implemented bi-directional websocket streaming of simulations character by character
- Built run archiving for replay and comparison
Stack
Python, AI Agent Development, LLM Prompt Engineering, React, FastAPI, WebSockets, LangChain, Groq, SQLite, LLM Evaluation
Case study: https://www.siddhvasudev.com/projects/odeon