Rohit Bhansali is Vice President of Software Engineering at Goldman Sachs.
Banks are investing heavily in artificial intelligence, but the more important question of “How much are they actually getting back?” is becoming increasingly difficult to answer.
It is not enough to know that teams are experimenting with AI, building proofs of concept or deploying new tools. Leaders need to understand whether the output returned by AI is valid, what guardrails should exist around its use, what testing endpoints are required and, ultimately, how much return the organization is receiving from its investment.
For banks, the goal should be to connect the resources AI consumes with the productive outcomes it generates, comparing its performance with the traditional alternative and ensuring its output is valid.
Start With The Cost Of AI Resources
The first question is relatively straightforward: What does the AI actually cost to operate?
Banks need visibility into resource utilization, including how many tokens are being used and how much compute is being consumed. They also need to know where those resources are being used and whether that usage is producing anything meaningful.
From there, the organization can begin comparing AI against the alternative.
For example, if I am using AI for software development, one way to think about ROI is to ask how many man-days or man-months it has saved compared with a person performing the same work. First, understand the resource cost, then measure the output against the human alternative.
That creates a much more practical starting point for evaluating AI ROI than simply tracking how many employees are using AI or how many AI applications a bank has launched.
Measure What AI Actually Changes
Ultimately, an AI investment should create an economic advantage.
If I am spending a certain amount of money on AI resources and getting more output than I would have received by spending that money on a person, the return is beginning to become visible. The same is true if AI enables the organization to complete work faster.
In the simplest terms, if I am reducing cost or getting my work done faster, my returns are better.
Speed can also have value beyond productivity. In certain parts of banking, including trading platforms, operating faster or achieving better results may create additional opportunities.
Did costs decrease, work get done faster or new opportunities emerge? Those are business outcomes, not AI activity metrics.
Activity Is Not The Same As Production Value
This distinction becomes especially important because banks can consume significant AI resources without creating corresponding value.
There may be teams doing testing, developing proofs of concept or running AI applications in the background. Those activities can consume a lot of compute and other resources even when the technology is never deployed into production.
Banks need governance that allows them to see how many tokens are being consumed, how much compute is actually costing the organization and whether that AI is really being utilized.
If an application remains in a nonproduction environment while consuming significant resources, leaders should ask: Where is it being used? Is it really needed? What productive outcome is it creating?
Experimentation is necessary, particularly while AI is still evolving. But experimentation itself should not be confused with ROI.
ROI Depends On Whether The Output Is Correct
There is another important dimension to AI ROI that traditional productivity calculations can easily miss, and that is validity.
How do we calculate whether the output returned by AI is actually valid? What guardrails should exist around that output? What testing endpoints should we have?
Ideally, organizations should also have an automated process for determining whether what AI has returned is correct. If AI produces work faster but employees have to spend substantial time validating, correcting or redoing it, the true return may be much lower than the initial productivity gain suggests.
This is particularly important in banking, where the consequences of an incorrect output can extend well beyond wasted employee time. AI ROI therefore should measure not only speed and cost but also the quality and validity of the output.
Governance Is Part Of The ROI Equation
Governance is therefore closely connected to ROI. Banks need guardrails around what AI is allowed to do and what it is not allowed to do.
For example, AI should not be able to use personally identifiable information however it likes. There needs to be guardrails protecting personal information and defining how data can be accessed and used.
There should also be limits on the actions AI can take independently. AI should not necessarily be able to create resources on its own unless someone has specified or approved that action. In other words, there are situations where a human should remain in the loop to authorize what the technology is doing.
To evaluate AI accurately, banks need to know what it consumes, what it produces, whether the output is reliable and what authority the system has to act.
Measure Outcomes, Not AI Activity
Banks should also recognize that AI ROI will not always appear immediately.
Sometimes the return may take time because the technology itself is still evolving. It may take time for an AI system to generate results that are better than what an expert in a particular field can produce.
Organizations may give up on some pilots too quickly, before they mature enough to produce meaningful value. But experimentation should still be subject to continuous measurement.
How much are we spending on AI resources? Where are those resources being consumed? What productive outcome are we receiving? Is the output correct? And are we completing the work more efficiently than we would using the traditional alternative?
As AI becomes more deeply embedded across banking, those questions will matter more than adoption numbers alone. The banks that develop a disciplined way to answer them will be in a much stronger position to distinguish between AI activity and AI investment that actually produces a return.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?