Samir Dutta, CEO and Co-Founder of Farsight AI.
For the past two years, enterprise AI leaders have obsessed over which model is best. They’ve been asking the wrong question.
Organizations have compared benchmark scores, reasoning capabilities and response quality to determine which AI model is smartest. That made sense when foundation models were rapidly differentiating themselves. Today, models are advancing so quickly that raw intelligence is becoming less of a differentiator and more of a commodity.
As models converge, picking the “smartest” one becomes less of a competitive advantage. The organizations seeing the strongest returns from AI are not necessarily using the smartest model. They are applying their chosen model more effectively to real business problems and pairing it with expertise competitors cannot easily replicate.
Yet many organizations are still measuring AI using yesterday’s framework. The next phase of enterprise AI requires a measurement framework built around business outcomes, operational impact and return on investment rather than model performance alone. As AI becomes more deeply embedded in business operations, the way you measure success needs to evolve alongside it.
Why Yesterday’s AI Metrics No Longer Work
As organizations expand their AI investments, demonstrating ROI has become a top priority for business and finance leaders. The challenge is that very few organizations have a consistent way to measure it.
Most organizations begin with time savings because it’s relatively easy to quantify. If AI helps employees complete work faster, organizations can point to higher productivity and reduced labor costs as initial benefits. But while that approach works well for repetitive tasks, it’s an incomplete measure of the value AI adds to core business processes, like improving proposal win rates, accelerating financial analysis or increasing customer retention.
Executive conversations are shifting beyond cost reduction. Leaders are asking whether AI can help win more business, reduce dependence on outsourcing, accelerate decision-making or create a lasting competitive advantage. Those are fundamentally different questions, and they require a different way of measuring success.
How To Build An AI Measurement Framework Around Business Value
No single metric can capture AI’s total impact on your organization. Instead, you should evaluate AI across several complementary dimensions that together provide a more complete view of business value.
1. Define success before you deploy AI.
Before introducing AI into a workflow, decide what success actually looks like. That sounds obvious, but many organizations deploy AI broadly and only begin thinking about ROI after implementation. Without clearly defined objectives, nearly any result can be interpreted as success or failure.
Instead, establish one or two business metrics that directly align with the problem you’re trying to solve. For example:
• Sales: Measure proposal turnaround time or win rates.
• Customer Service: Track case resolution times or customer satisfaction.
• Financial Analysis: Evaluate how quickly analysts produce investment recommendations without sacrificing quality.
When every AI initiative is tied to a measurable business objective from the beginning, demonstrating value becomes significantly easier.
2. Evaluate whether AI improves the final work product.
Productivity gains only matter if the work AI produces is good enough to move the business forward.
In financial services, we measure something that deserves much more attention across industries: acceptance rate. We look at how much of the work produced by AI ultimately makes it into the final presentation, financial model or client deliverable.
That distinction matters because work that’s only 60% usable often has very little value. If employees have to discard almost half the output and start over, the anticipated productivity gains quickly disappear.
Whether you’re generating proposals, reports or customer communications, evaluate whether AI is consistently producing work your employees trust enough to refine and deliver. That provides a much clearer indication of whether AI is delivering meaningful business value.
3. Measure how effectively AI leverages your institutional knowledge.
As foundation models continue improving, they’ll grow increasingly capable across a wide range of business tasks. That makes your organization’s expertise even more valuable. Rather than relying primarily on publicly available information, prioritize connecting AI more deeply to the knowledge that already exists inside your business.
For instance, AI may have access to historical project work, internal methodologies, proprietary research, customer data or operational playbooks. But simply making that information available isn’t enough. The more effectively AI incorporates those assets into its recommendations and work products, the more likely it is to produce outputs that competitors can’t easily replicate.
One question worth asking is how much of the information your AI uses comes from your own institutional knowledge versus publicly available sources. The more effectively you incorporate your organization’s expertise into AI-powered workflows, the more differentiated the outputs become.
When Intelligence Is No Longer The Differentiator
Every major technological shift eventually reaches the same point. The technology itself becomes widely available, and competitive advantage shifts to how organizations use it.
AI is rapidly approaching that moment. The organizations that create lasting value won’t spend the next several years chasing every new model release. They’ll focus on building the systems, processes and institutional knowledge that allow AI to consistently improve business outcomes.
That’s ultimately what I believe executives should be measuring. Long after today’s models have been replaced, the organizations that know how to apply AI effectively will continue to outperform those that simply adopted it.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?