Voice AI Debt-Collection Agent With Full-Duplex SIP Calls
AI products · Creator, sole engineer · shipped
An outbound voice agent that places real phone calls, negotiates in real time, and can be interrupted mid-sentence like a real conversation.
Watch: https://www.youtube.com/watch?v=VsEfOfwh8XM
How it works
Outbound calls over SIP. When a call ends, a 70B model scores risk, legal flags and the outcome.
- Caller → LiveKit
- LiveKit → Deepgram STT
- Deepgram STT → Groq LLM
- Groq LLM → Deepgram TTS
- Deepgram TTS → LiveKit
- Groq LLM → Risk review
The detail
A risk engine analyses each call's transcript as soon as it is saved. It flags regulatory triggers - bankruptcy, legal representation, cease-and-desist - classifies the person's situation, and tags the outcome as a payment promise, refusal or dispute.
A dashboard shows each call's transcript with its risk indicators and aggregates recovery rates across calls.
Built as three deployable services so the voice agent's event loop never shares a process with the API.
Python, FastAPI, LiveKit, Deepgram, Groq, React, Docker.
What I did
- Built the complete system - voice agent, API, risk engine and dashboard
- Implemented outbound telephony over SIP with full-duplex interruption
- Built the compliance engine that flags regulatory triggers from each call's transcript
- Built automatic outcome classification from the conversation itself
- Architected it as three separate services so the voice event loop never shares a process with the API
- Built the dashboard showing transcripts with risk indicators
Stack
Voice Assistant, Python, Speech Recognition, AI Agent Development, FastAPI, LiveKit, SIP, Groq, Deepgram, Docker, Real-Time Systems
Case study: https://www.siddhvasudev.com/projects/vaani