Why Your AI Transformation Might Be Failing Before It Begins

Every technology wave starts with the interface and misses the point.

Over the past three decades, technology leaders have lived through several technology transformations: ERP, CRM, SCM, HCM—we replaced paper trails, spreadsheets and home-grown databases with integrated enterprise systems, expecting technology itself to create business value. It didn’t.

Should we blame technology for that? No. The problem was where we focused our attention.

Organizations became obsessed with digitizing how work happened while forgetting to understand why it happened in the first place. Subject-matter experts were rarely asked to explain the rationale behind processes. Instead, companies either automated existing ways of working or adopted a vendor’s definition of “best practice.” In both cases, the context disappeared.

Then came the cloud era. Customization gave way to standardization. Organizations accepted more vanilla processes in exchange for faster innovation and lower operating costs. Yet the fundamental problem remained unchanged: The “why” was still missing.

This is the reason that, years after completing cloud transformations, many organizations still struggle to articulate the business value of their enterprise applications. It’s also why users continue describing their systems as fragmented, unintuitive and, ultimately, clunky.

Now AI has arrived, and organizations are once again making the same mistake by starting from the output instead of the input.​

AI cannot guess the context; it will only approximate it.

​Consider something as simple as prompting. Thousands of videos showcase the “ta-da” moment where an AI model produces an impressive response in seconds, but far fewer explain the ingredient that actually determines the quality of that response: context.

Anyone who’s taken a basic course in human communication understands why this matters. Imagine asking someone abroad to recommend a restaurant without mentioning your budget, dietary restrictions, cuisine preferences or the occasion. Would you expect a perfect recommendation? Of course not. Yet that’s exactly what many users expect from AI.

Large language models cannot read minds. When context is missing, they don’t magically infer it—they approximate it. They draw on patterns learned from vast amounts of public data and generate what is statistically most likely to be useful. Sometimes that’s enough. But the more organization-specific the task, and the higher the stakes, the greater the probability that AI will confidently produce something that misses the mark.

The same pattern is now scaling across the enterprise. Organizations are investing billions in AI. Survey after survey tells the same story: Experimentation is widespread, and investment is accelerating, but measurable business value remains elusive. The gap between enthusiasm and impact remains stubbornly large.

Why? Because many organizations are implementing AI at the very top of the technology stack—the experience layer—while leaving the underlying business context fragmented across disconnected systems, siloed databases and undocumented knowledge. Users then conclude that AI isn’t accurate enough. The problem is, again, about the context. It’s like renovating the façade of a house while leaving every room inside cluttered and disconnected. The exterior may look modern, but living there stays the same.

Start with the context to harvest value from AI.

If organizations want AI to become transformational rather than merely impressive, starting with context is essential.

Some of that context already exists inside enterprise applications. Customer histories, financial transactions, workforce data, operational records—these systems contain enormous amounts of organizational knowledge. The challenge is that this knowledge is often fragmented across dozens, sometimes hundreds, of applications.

In one of my previous global roles, I counted more than 150 enterprise systems, including over 30 separate learning platforms. Fragmented databases, redundant applications, unnecessary integrations and unclear support models—all these created silos where context became trapped and posed risks for reaping value from transformation. In contrast, in my most recent role, building AI applications on top of a single HCM data model gave us a much more consistent foundation for those applications, significantly accelerating deployment and adoption while making the outputs easier for users to trust.​

Long story short, system and data consolidation shouldn’t be a nice-to-have. Treat it as essential, and be ruthless with it.

But structured enterprise data is only part of the picture. Equally valuable context lives outside transactional systems—in digital workplace platforms, project discussions, meeting notes, documents, decisions, conversations and, too often, only inside people’s heads. This organizational memory is frequently invisible to AI because it has never been systematically captured, evaluated or connected.

Before you start redesigning experiences with AI, take a step to understand how organized the knowledge those experiences depend upon is. In one initiative of my team to combine multiple stakeholder perspectives in a single market sensing capability, data consistency was the step that took the most time, not building the AI models to analyze the data. As we add more data sources to the mix, this is the main criterion to observe.

The winners in the AI era won’t be those with the most sophisticated models or the most elegant conversational experiences. They’ll be the organizations that understand their own business better than anyone else and can make that understanding available to AI. Because AI doesn’t create context. It amplifies it. And that’s why every successful AI transformation starts with context.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?