voiceops
The deployment kit I extracted from my first voice agent, proven by taking a second, different customer live with it.
Overview
voiceops is the reusable part of Slotback, my voice-agent deployment for a fictional hospital: safety defaults for the agent, a release gate, a tamper-evident ledger, deploy checks and a review sheet for real calls, as two open-source Python packages. I extracted only what Slotback already used and moved Slotback onto it with no change in behaviour. Then I tested the kit the way the field would: by taking a second, different customer from an empty folder to a live agent, simulated conversations and real phone calls.
At a glance
- 1 h 17 minfrom an empty folder to a second live agent passing its release gateMeasured · one build session
- 0 linesof kit code copied into the second deployment, which wrote 302 lines of its ownMeasured
- 6 releasesof the kit, each forced by something the second deployment foundMeasured
- 403 of 403tampered ledgers judged exactly as Slotback’s original verifier judged themMeasured · parity test
- 12/12release gate on the second deployment: 9 tool-level checks, 3 simulated conversationsSimulated conversations
- 3 callsreal calls to my own phone; each one found a problem the simulations had notReal calls
The problem
A voice agent that talks to the public needs the same safeguards every time: say it is an AI, verify identity before any detail, hand over to a person on request, promise nothing no system does, and leave a record someone else can check. On my first deployment, Slotback, those safeguards took most of the engineering.
Rebuilt for each customer, they drift, and the second customer gets a weaker copy of the first. The usual answer is a framework designed up front, but designed before a second customer exists, it can only guess which parts are general.
The approach: extract only what one real deployment already used, prove the first customer behaves exactly as before, then let a second, different customer decide what the kit is missing.
What I built
Two Python packages, open source and on PyPI, driven by one config file.
- voiceledger: a hash-chained, Ed25519-signed event ledger with pluggable rules, such as “no detail before identity is verified” or “every emergency handed to a person within 120 seconds”, and a verifier an auditor installs on its own: 4 packages, 12 MB.
- voiceops: a command line for the whole deployment. init scaffolds a project; build composes the agent’s prompt from the project’s own fragments plus seven safety defaults that cannot be dropped silently; push publishes the agent; gate is the release gate; check runs deploy checks against the live API; call, fetch and review place a real test call to one allow-listed number and turn it into a review sheet.
- A guard for the API the agent calls: per-call tokens bound to one verified person, signed webhooks, rate limits and a log allow-list. Unauthenticated calls are refused before any input is read.
My role
I treated the kit as a product with one customer already live and a second one to win.
- Scope and success criteria. Five criteria fixed before the build: no behaviour change for Slotback; a second deployment live and passing its gate in three working days or less; mostly domain code; a kit that tests itself; an auditor verifies with one small install. One rule: extract only what Slotback already uses. Plan ↗
- Extraction with proof. Slotback moved onto the kit one module at a time, each compared with the original on Slotback’s own code and data before it merged. Results ↗
- A second customer. claims-surge, a fictional insurer’s claims line after a storm: claim status, new claims, and emergencies passed to a person before any identity check. claims-surge ↗
- Field testing. Live on Render, agents pushed to ElevenLabs, ten simulated-conversation runs and three real calls, each reviewed line by line. Time log ↗
- Back into the product. Every finding became a kit release with a test, or a written decision. Decision log ↗
Key design decisions
Where the judgement is.
Extract after the first deployment, not before.
The kit holds only what Slotback used. What to add next was decided by the second deployment, not by opinion.
Safety defaults you can reword but not lose.
Seven safety fragments: AI disclosure, upset caller, identity before detail, voicemail, a person on request, no unbacked promises, closing. A project may reword one, or remove it with a written reason. Dropping one silently fails the build, and every agent comes with a manifest of how each is met.
Our checks decide; the platform’s judge advises.
The gate runs its own deterministic checks on every simulated conversation, including a privacy-leak detector that must catch every planted leak before its “0 leaks” counts. The platform’s judge passed an agent saying “I’m connecting you” when the tool only books a callback; our check failed it.
Rules as data.
Ledger rules are four generic types set in config. Slotback’s two checks became rules unchanged; the claims line added two that Slotback never needed.
Never touch another customer.
Two deployments in one platform workspace share tool names. The push matches tools by the endpoint they call, never by name, checked against the live workspace with Slotback’s 13 tools unchanged.
Real calls are part of the release.
Calls go to one allow-listed number only. The review sheet masks phone numbers, flags tool errors and promises, and leaves root cause and fix to a person.
What the second customer found
Each finding, a release.
Slotback alone could not have found these. Each one is a kit release with a test.
0.1.1 · The push
Found
Pushing the second agent would have repointed or deleted Slotback’s live tools: both projects have a tool called log_outcome.
Changed in the kit
The push only adopts tools that call its own endpoint.
0.1.2 · The platform
Found
The platform refused an English agent on a multilingual voice model.
Changed in the kit
The build fails first, in CI, where it is cheaper to learn.
0.1.3 · The judge
Found
The agent said “I’m connecting you” when the tool books a callback, and the platform’s judge passed it.
Changed in the kit
A forbidden-phrase check on every agent turn; the handoff default says it is a callback.
0.1.4 · Our own check
Found
A correct refusal, “I can’t tell you whether it’s covered”, failed the gate on a substring match.
Changed in the kit
Phrases can be patterns that skip refusals, with each real sentence kept as a known answer.
0.1.5 · Real call 1
Found
The API restarted as the call began; the agent told a caller with the right details that they didn’t match.
Changed in the kit
A tool error is “a technical problem”, never the caller’s fault; the review sheet counts error replies.
0.1.6 · Real call 2
Found
The caller’s spoken number appeared in full on the review sheet, and an example date the agent improvised was repeated back as an answer.
Changed in the kit
Every phone-like number is masked; the agent never gives example answers.
Real call 3 found a claim registered with a year the caller never said. That fix belonged to the claims line, not the kit: its server now checks the date.
Hear it
A real call.
Calls go only to my own allow-listed phone. I play a synthetic policyholder on the fictional Contoso Insurance claims line.
AI disclosure, identity check, safety question, read-back, claim registered and reference spelled out. It sounds right, and it was wrong: I said “October 15”, and the agent registered 15 October 2024, two years back. No simulated run had caught it. The server now refuses a date with no year, in the future or over a year old, and the agent reads the date back with the year before registering.
Results against the plan
| S1 · Slotback on the kit with no behaviour change: same verdicts on 403 tampered ledgers, byte-identical agent configs, 117 of 117 guard tests, live checks 6 of 6 | Met |
|---|---|
| S2 · Second deployment live and passing its gate within 3 working days: live at 1 h 07 min, gate at 1 h 17 min, first completed real call reviewed at 17 h 54 min (overnight) | Met |
| S3 · Mostly domain code: 302 lines of its own on 2,841 lines of kit, none copied | Met |
| S4 · The kit tests itself: 167 tests, known-answer sets for the ledger and leak detector, init output gated in CI | Met |
| S5 · Independent verification: the verifier installs alone, 4 packages, 12 MB | Met |
Simulated conversations cost 15,901 credits over ten runs. Each failing run found something real or tightened a check, and every run is logged with its finding and fix.
Architecture
Tech stack
- Python 3.12
- uv workspace
- pydantic
- Typer
- FastAPI
- PostgreSQL (psycopg 3)
- SHA-256 hash chain + Ed25519 signatures
- ElevenLabs Agents (agents, tools, Tests API)
- Twilio
- pytest + Hypothesis (167 tests)
- Ruff
- GitHub Actions
- PyPI
- Render (EU)
What I’d do next
- Replace the phrase lists with a calibrated check: three of the ten runs failed on a phrase that caught a correct refusal.
- Let the handoff use the number an outbound call dialled, so “call me on this number” is enough.
- A third deployment in Spanish, to test the multilingual paths the claims line did not use.
- Publish from GitHub Actions with trusted publishing instead of a stored token.
Fictional customers (a hospital in Slotback, Contoso Insurance in claims-surge) and synthetic data only. Real calls go only to my own allow-listed phone. Conversation-level results are simulated on ElevenLabs Tests and labelled as such; everything else is measured. ElevenLabs is the voice-agent platform used; this is an independent project, not endorsed or sponsored by ElevenLabs.
Next projectContract continuity on n8n