Alexandro Picolini is a technology executive and software architect focused on telecom, cloud and AI.
Voice AI demos have become surprisingly easy to build. With a real-time model, a microphone and a good prompt, a team can create something impressive quickly. But a demo is usually a controlled environment: clean audio, a stable network, cooperative users and readily available data.
A real telephone operation is very different.
I am a CTO and software architect with more than 25 years of experience in telecommunications and software development. My company develops contact-center platforms that connect telephony, digital channels and customer management. Our work integrating voice AI into these environments has shown me how much production depends on systems beyond the model.
The AI Only Hears The Audio It Receives
One of the first problems we faced was telephone audio quality. As a call passes between carriers, its audio may be compressed and converted along the way. Receiving it in a higher-quality format at the final connection does not restore detail already lost. A person may still understand the conversation while a speech-recognition system struggles.
Speakerphone calls add background conversations, television and traffic. Voice activity detection—the technology that detects when someone is speaking—does not always distinguish the caller from surrounding voices.
It’s important to test through the actual telephone routes customers use, not only a headset connected to a computer. Teams should include mobile phones, speakerphones and noisy environments. They should listen to the audio reaching the agent, not just the voice it generates.
Turn-Taking Is Still Difficult
Human conversations include brief acknowledgments: “Uh-huh.” “Okay.” “Right.” These do not always mean someone wants to take over the conversation. If interruption detection is too aggressive, a simple “okay” stops the agent mid-explanation. If it is too conservative, customers must speak over the agent before it stops.
In some of our implementations, around two seconds has been a workable compromise. It is not a universal setting: Turn detection requires more than detecting sound.
It’s critical to test acknowledgments separately from deliberate interruptions. The goal is not to eliminate every overlap but to let customers interrupt when they need to without making ordinary listening behavior break the conversation.
API Latency Feels Very Different On Voice
An API taking three or four seconds during a WhatsApp exchange may barely register. On a phone call, those same seconds feel long.
You can ask customers to wait or provide subtle audio feedback. We have experimented with sounds that simulate typing while information is retrieved. That may work once; repeated pauses quickly become frustrating.
Whenever possible, information the agent is likely to need should be available through a fast data layer. In some implementations, we use Redis to cache customer information and reduce slow backend requests.
Teams should identify which lookups must happen during the call and which can happen beforehand. They should set a waiting limit for each dependency and decide what happens if it fails. A reassuring phrase is not a substitute for a responsive integration.
Customer Context Should Not Be An Afterthought
When someone shows me a voice AI demo and says it is production-ready, I ask: “How do you dynamically deliver the contact’s data to the bot?” The system may already know the campaign, customer account, recent interactions or reason for calling. Making the customer repeat everything wastes that context. Retrieving each detail only when requested adds delays.
Start by defining the minimum information needed for each use case. Pass it into the conversation through a consistent structure rather than loading an entire customer record. Separate background information from data that must be refreshed before an action, such as confirming an appointment.
Define what the agent should do when a record is missing, outdated or inconsistent. Restrict information to what the task requires, and verify identity before disclosing sensitive account details. A recognized phone number should not, by itself, authorize disclosure. Test those failure cases as deliberately as the ideal scenario.
Human Handoff Should Be Designed From The Beginning
Human transfer is not merely a rescue mechanism for a failed AI. Customers may request a person, a situation may exceed the agent’s scope or an action may require human authorization.
Define these transfer triggers with the operations team before launch. Specify the destination queue, its operating hours and an alternative when nobody is available. Test busy queues, unavailable agents and failed transfers—not just a successful connection. The transfer should carry a concise summary: why the customer called, what has been verified, actions already attempted and what remains unresolved. Make that information available where the human agent actually works.
Do not tell the caller that a transfer succeeded merely because the system requested it. Confirm the outcome, explain any delay and offer the agreed alternative when necessary. The customer should not lose the conversation’s progress at the moment human help becomes necessary.
Scale Exposes Problems That Demos Hide
Many problems do not appear with one or 10 simultaneous calls. At thousands of concurrent calls, we have seen delays in audio delivery over SIP that were almost invisible at small scale. It’s important to test the complete path under realistic concurrency, including external APIs and human transfers. Measure the time callers wait to hear a response, not only the time a model takes to generate one. Include slow dependencies and provider interruptions in those tests.
A polished demo may prove that the model is capable. It does not prove that every connected system will remain responsive under load. A good demo proves the AI can talk. Production proves whether the whole system actually works.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?