Amirtha Saminathan is a data and analytics leader specializing in scalable platforms, data governance, and AI-driven decision-making.
Plenty of companies have an AI assistant that launched with a lot of noise, yet sits ignored today. There were slides, there was a demo everyone loved, and then, a few quarters later, someone stopped pulling the usage numbers and the whole thing faded. Nobody ever wrote an honest note about why.
When AI assistants fail, everyone wants to blame the model. It wasn’t smart enough, it made things up, pick a reason. But the model is rarely the culprit. MIT’s 2025 NANDA study found that 95% of organizations are getting zero return on their generative AI investments, and concluded the divide is determined by approach, not model quality or regulation.
That matches what I’ve seen. The assistant gets built like a feature when it needs to be built like a system, and the company was never ready to run it once the pilot ended. At Fortune 50 scale, where it’s fielding millions of customer conversations, that mismatch stops being an inconvenience and starts costing real money.
What The Pilot Doesn’t Tell You
When I led the implementation of an AI assistant, the goal was to handle a big volume of customer questions and plug into the systems that run daily operations. I spent my energy bracing for the language model. It behaved fine the whole way through.
The pilot went great, which is exactly what fooled us. Clean data, a controlled setup, barely any dependencies. But a pilot looks good precisely because someone has already removed the hard parts, and production puts them all back. The day we connected to real systems, we were pulling from a dozen sources that refreshed on totally different schedules, some by the minute, some once a day. The assistant was answering with a straight face using data that was already hours stale.
Nothing was wrong with the model, only with everything it relied on.
Fluency Is Not The Same As Trust
In a consumer app, a smooth answer is usually good enough. In a regulated business it isn’t. An answer you can’t trace, can’t explain and can’t limit to what a given person is allowed to see isn’t helpful. It’s a problem waiting to happen.
A huge chunk of the work went into things that never make a demo reel: teaching the assistant who the user is and what they’re cleared for, logging enough to reconstruct how it landed on an answer, and fencing it in so it wouldn’t get creative in the middle of a transaction. None of that is glamorous. All of it is the difference between something you can put in front of customers and something you can’t.
The Value Lives In The Plumbing
Here’s what surprised me most: Almost none of the value was in the conversation. It was in the integrations. The assistant only earned its keep when it could reach into systems of record and actually fix something, not just narrate the problem back to you. An enterprise assistant sits on top of your platforms and coordinates them. If it can’t do that safely, all you’ve built is an expensive way to ask questions.
Whether people used it had almost nothing to do with how well it was designed. A good assistant still dies if the answer comes too late or doesn’t fit how someone already works. Getting to anything measurable meant getting engineering, finance, compliance and operations to agree on what “working” even meant, which was harder than any of the technical work. The tech was ready months before the organization was, and that gap decided whether the whole thing was worth the spend.
What I’d Tell Any Leader Greenlighting An AI Assistant
If you’re the one signing off on an AI assistant, this is the part worth slowing down for. This is where these projects actually live or die.
Design For Production From The First Day
If you treat the first version as a throwaway and plan to add security, monitoring and real data standards later, then you’re not scaling anything, you’re just rebuilding it from scratch.
Build your AI assistant from day one to survive in the environment it’ll actually run in, even if it looks less impressive early.
Sort The Data First
Teams tend to chase one more point of accuracy while the pipeline underneath is held together with tape. Boring and reliable beats slightly smarter every time. Most of what gets called an “AI problem” is a data problem or an ownership problem wearing a costume.
Decide Who Is Accountable Up Front
Who watches performance? Who answers for it when it’s wrong? Who explains a decision when a regulator or a furious customer asks? When nobody can answer questions like these, the organization gets skittish about scaling, and that kills more projects than any bug ever will.
Put A Human On The High-Stakes Calls
Running everything fully automated on day one is how you get a very confident mistake at scale. Work out which interactions can stand on their own and which need someone to step in, then design those handoffs so the assistant hands the human real context instead of a cold start. The goal is not maximum automation; it’s automation you can stand behind.
Measure What It Actually Shifts
Measure what actually shifts, not whether it’s live, because getting it live is the easy part. The real question is whether it’s cutting genuine effort, speeding up real decisions and leaving customers better off. Something that demos beautifully but moves none of those numbers is a science project, not a business system.
The Real Dividing Line
The companies that get this right aren’t the ones with the fanciest models. They’re the ones willing to treat AI like any other critical system—something you engineer, govern and staff—and to do the unglamorous work of getting the organization ready to run it.
That part never makes the press release. It’s also why some of these assistants last and the rest become ghost towns.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?