Anatomy of a voice agent on a real booking system

One call, four parties

A caller rings the club and asks for a tee time on Saturday. During office hours the call rings the club office first. Outside them, or when nobody picks up, it goes to the agent. The agent is a hosted voice model with a prompt, a knowledge base and a set of tools. Each tool is an HTTP endpoint on our middleware.

The agent never talks to the booking system directly. It asks the middleware to check slots. The middleware queries the booking API and returns free times. When the caller says "book it", the agent calls book_tee_time. The middleware checks the club's rules, requests a quote, places the booking and hands back a reference. The agent confirms. When the caller hangs up, the voice platform sends a signed webhook with the transcript and its analysis of the call.

Each voice session is a separate, stateless agent instance, so concurrency pressure lands on the middleware, not on the model. It has to handle parallel booking requests without races, and booking writes are guarded against replays, so a retried request returns the earlier booking instead of making a second one.

The middleware holds the rules

The middleware is a Node.js and Express service in strict TypeScript, backed by Postgres. Every tool request is validated with Zod before anything else happens. The first version was a thin wrapper around the booking API. It did not stay thin, because club rules kept turning out to live only in the agent's prompt.

The advance-booking window is the clearest case. Guests and members may book a different number of days ahead, and one identified member grants the longer window to the whole flight. Tested live, a guest on their own could book inside the member window: the prompt said no, the API said yes. A comment in our code puts it plainly:

A prompt line is a suggestion to a model; this is the gate.

Other rules moved into code the same way:

• Cancellation deadline. A cut-off on the evening before the tee time, computed in Vienna wall-clock time so it holds across daylight-saving changes.
• Same-day duplicates. If the caller already holds a slot that day, the booking returns a soft conflict with a read-back line, and the caller has to confirm. Two rounds in one day is legitimate, so this asks rather than blocks.
• Refusals the caller can hear. When the booking system rejects an order, for example on a handicap rule, the middleware turns the error code into a sentence the agent can speak, using the club's own wording. Failures that are ours, such as an account limit, get a neutral line for the caller and an alert for us.
• Prices that match. Member rates are only computed at the quote step, for players who carry an association membership number. The price tool runs the same preparation code as the booking route, so the figure the agent speaks is the figure that gets charged.
• References a person can write down. The first booking reference was a 40-character string. It is now a short prefix plus six characters from a 25-symbol alphabet with the pairs that collide on a phone line removed (0/O, 1/I/L, 5/S, 8/B, U/V), collision-checked against the database.

Time zones earned their own module. The booking API returns local times with no zone designator. Stored as-is in a timezone-aware column on a UTC server, every tee time was off by the Vienna offset, and the dashboard showed bookings two hours late. The fix parses club times explicitly as Vienna time, and a migration repaired the historical rows.

There is also a kill switch. One environment variable makes every voice-tool endpoint return a friendly 503, which the agent relays as "please call the office", while health checks, admin routes and the post-call webhook keep running.

After hang-up: the post-call webhook

The webhook is where a call becomes a record. The handler reads the raw body, checks an HMAC-SHA256 signature over the timestamp and body with a constant-time comparison, and rejects anything outside a five-minute clock-skew window. Then it writes one row to a calls table and one row per evaluation criterion to an evaluations table. A per-call summary email goes to the office, and an urgent escalation goes out if the call was marked high urgency.

Three details here came from things going wrong:

1. Model output is not trusted to fit the schema. Three columns only allow a fixed set of values. The model once answered a member-status field with a value outside that set, the insert failed, and the call was lost: no row, no email, nothing on the dashboard. Now out-of-range values fall back to a safe default, the full payload is kept on the row, and the drift is logged.
2. Telephony beats transcription. The caller's number comes from the telephony metadata first and the model's extracted field second. Placeholders such as "unknown" or "anonymous" are stored as empty, not as a phone number.
3. A mail failure never fails the webhook. The call row is already saved when emails go out. If the handler returned an error, the platform would retry, re-send whatever mail did get through and, after enough failures, disable the webhook. So mail errors are logged and the webhook is still acknowledged.

Seven criteria and eleven fields

The voice platform analyses every conversation against criteria we define, and extracts structured fields from the transcript. This agent has seven evaluation criteria and eleven extracted fields. They are the difference between "we have recordings" and "we know how the agent is doing".

The criteria check behaviour: did the agent greet properly, stay in the caller's language, avoid inventing facts, handle membership pricing and handicap rules as instructed, offer a handover when it should and handle personal data as instructed. Each result is a pass or fail with the model's reasoning. They are graded by a model, so they are a signal for review, not a verdict.

Criteria are stored generically, one row per call and criterion name, so they can be added or renamed without a schema change. The dashboard turns them into a daily share of criteria passed across all calls, so the effect of a prompt change is visible day by day.

The fields describe the call: language, inquiry category, member status, urgency, caller name and phone, the action requested, key points, and three flags for satisfaction, transfer and callback. There is no booking-reference field, which mattered. The obvious link from a booking to its call, our internal call ID, cannot work: that row only exists after the webhook, when the booking was made minutes earlier. The fix passes the platform's conversation ID into the booking tool as a variable during the call, and the dashboard joins on it.

The callback queue is the way to a human

Some calls the agent cannot finish. At the club's decision, live transfers were dropped, which makes the callback queue the only route from "the agent could not finish this" to a person.

An entry has a caller name, phone, reason and urgency, and gets there in one of two ways:

• The agent calls send_callback_request during the conversation. Urgency defaults to medium.
• An optional net in the webhook handler catches a specific failure: the agent spoke, the caller never did, the call ended within twenty seconds, and we know the number. That caller probably heard silence. The handler opens a high-urgency callback, at most one open entry per number.

The queue sorts high before medium before low, and within a level the oldest request comes first. Closing an entry sets done_at and done_by. Only open rows can flip, so a double click changes nothing. Before launch, test-era entries were closed through a migration that names itself as the closer, so the history survived and the queue showed its true state.

What the club office sees

The dashboard is a separate Next.js app reading the same Postgres database the middleware writes to. It is built for the office, in German, not for us.

Access for this first version is one shared staff password, exchanged for a signed HttpOnly cookie with a twelve-hour lifetime. Changing the password ends every session at once. Per-user accounts are the upgrade path if the team using it grows.

What building it taught us

The agent's prompt and tool configuration are tuned by Steven Gorea; the middleware and dashboard are ours on the engineering side. Most of the hard problems sat on the seam between the two.

• Put rules in code. Anything that must hold, such as booking windows, deadlines or prices, belongs behind an endpoint the model cannot talk its way past.
• Assume the model writes outside your schema. Keep the raw payload, coerce to known values, and never let a bad field lose a call.
• Roll out across the seam in order. Endpoint authentication is built to run warn-only first and to be enforced only after every agent tool sends the header. Enforcing early would take the agent off the air.
• Plan for data leaving. A daily job at 03:00 Vienna time deletes call rows and their evaluations after a configurable window, 30 days by default, and logs each run. Erasure requests have to reach the recordings held by the voice platform too, not only our own tables.

None of this is visible to the caller, who hears a voice that knows the tee sheet. That is the point. The voice is the smallest part of the system. The rest is ordinary, careful backend work, and it decides whether the office trusts what the agent did.