Rizwan Jan is VP and chief information officer of The CNA Corporation.
In my industry, I often see companies implement AI in a fragmented way. Departments like finance and HR each use their own native AI system. Each one works, but the data stays siloed. Nothing is bridging the two systems except people: Someone has to extract data out of the HR system and manually hand it off to another in finance, who then has to upload it to their system.
AI without integration just creates more fragmentation. When companies spin up models within their own departments or business units, they end up with pockets of employees using their own engines with their own dashboards. Ask what they’re doing with the new capacity, and the answer might be “getting coffee” or “having longer lunches”—none of which is driving business value at all.
AI models that never connect across boundaries within the organization only get a subset of the data they need to make informed decisions. Instead, companies should be asking how AI is actually driving business value. The answer is an orchestration layer that stitches AI systems together.
Have The Conversations First
Creating an orchestration layer requires restructuring how work gets done, not deploying yet another platform. The first step is building trust within the company. This means explaining why, how and where AI is going to be introduced and what problems it is trying to solve, as well as where human judgment and accountability are going to be retained.
It’s crucial to develop trust between those who deploy the technology and those who use it. Trust drives adoption, and a key part of that comes from having conversations before rollout. Too often, IT will push a system onto employees with little or no explanation. We first need to ask whether people actually need a new tool. How will it make their job easier? Is it going to provide more time or higher-value research?
Conversations should be part of a continuous feedback loop; they can tell you what’s working and what’s not before you roll out a system enterprise-wide. When we initially rolled out our proprietary LLM internally, the feedback was mildly positive, but many employees felt it wasn’t really making their jobs easier. When we asked more questions, we found that people didn’t know how to prompt the system correctly—meaning conversationally, instead of Google-style “What is X?” So we ran a working group session on prompt engineering, and results improved dramatically.
Peer Advocacy And Measured Outcomes
Peer-to-peer advocacy—a.k.a.“water-cooler talk”—can drive adoption of a technology better than any top-down mandate. When you hear about your colleagues using something effectively, you’re going to be more receptive than when IT is pushing a new technology down your throat. That’s why we launched a “champions program” of employees who can talk about using certain AI models in research tasks, and why and how others might consider using them.
It’s also important to focus on outcomes instead of adoption. Business units should measure their own results, whether that’s efficiency, cost or revenue, since they’re the subject matter experts on their own datasets. For us, this means examining whether AI is helping provide faster and more valuable research and whether it’s freeing up funds for other types of work and ultimately creating more capacity for researchers. Many companies are surprised when they actually start looking at outcomes and find they’re not what they hoped for.
Put The Human On The Loop
The idea of keeping a “human in the loop” comes up a lot with AI, and I agree oversight is critical. At the same time, I think we should shift this mindset to something more like “human on the loop.” We should aim for a point where AI is doing some or even the majority of the work while a human sits on top of that loop and only manages the exceptions, such as paying invoices above a certain threshold. That means removing people as the friction layer between native AI systems and putting in place an enterprise-wide orchestration layer instead.
We’re moving in the right direction, but it’s still early days; I’d estimate about a third of companies are implementing AI correctly, and the rest are using AI in a fragmented way. I’m starting to see the industry pivot toward integration platform as a service, though the handful of companies in this space handle integration without orchestrating from an AI perspective.
The opportunity lies in having AI systems that can work together across the enterprise, with people overseeing and measuring whether the results are actually creating business value. Technology enables change, but change management is what will make transformation stick.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?