Swati Deepak Kumar is Senior Vice President of Information Technology at Citi.
A CIO may see AI applications from different vendors and assume the enterprise has options. But what if the applications that appear to offer different options all rely on the same underlying AI models or infrastructure?
In my previous article, “Cognitive Concentration: The Hidden Risk Beneath Enterprise AI,” I explored that dependency. Here, I want to show how leaders can measure it and decide where resilience investment belongs. My central argument is that concentration alone does not determine risk. Leaders also need to consider the consequences of a failure and how easily they could switch to an independent alternative.
A 2026 IBM survey conducted with Oxford Economics found that 73% of 1,000 executives “describe their AI environments as intentionally multi-vendor,” yet 91% reported incomplete understanding of AI dependencies, and 71% said that “switching their primary provider or model would be difficult.” These are reported perceptions, not observed failure rates.
Six Questions Behind The Cognitive Concentration Framework
While developing my CIO dashboard, I identified a measurement gap: An application inventory can show which technologies an enterprise uses but not whether leaders can independently verify important AI-assisted decisions or keep those decisions moving when a system fails. I propose a Cognitive Concentration Index to help leaders assess these dependencies. It is a management tool for identifying potential weaknesses, not a predictor of failure:
• Concentration: What is shared? Map workflows to model families, APIs, compute, critical data and specialist talent. Calculate criticality-weighted concentration by dividing the sum of criticality weights of workflows dependent on an ecosystem by the total across assessed workflows. Measure model and compute exposure separately; overlapping shares need not total 100%.
• Criticality: What is the consequence? My proposed 1-to-5 scale emphasizes reversibility and blast radius. An editable draft might rate 1; a hard-to-reverse procurement commitment might rate 5. Business owners assign the scores.
• Substitutability: Can you switch? Record independent fallback coverage, tested recovery time, lost functionality and manual options. A backup model is an architectural option; a tested alternative decision path is a resilience capability. Mark untested fallbacks unverified.
• Change Exposure: What could shift together? Count consequential decisions and processes exposed to one upstream update, not just API calls.
• Validation: Can outputs be independently challenged? A 2025 study of more than 350 language models found that, on one benchmark, models chose the same wrong answer 60% of the time when both were wrong. For some decisions, leaders may get a more independent check from conventional analytics, explicit business rules or human review than from a second AI model.
• Decision: Where should you invest? Accept concentration for reversible tasks; fund fallback tests, staged changes or approval gates when consequences are material.
The Financial Stability Board’s 2026 sound practices address materiality, performance, oversight and third-party dependencies in financial institutions.
What A Six-Workflow Simulation Shows
To examine the dashboard’s logic, I use a synthetic comparison, not a real-enterprise case study. Six workflows—procurement, production scheduling, financial approvals, customer remediation, maintenance planning and sales insights—carry criticality weights of 5, 5, 4, 4, 3 and 3, totaling 24.
Procurement, financial approvals and maintenance use Model Ecosystem A: 5 + 4 + 3 = 12, or 50% criticality-weighted model concentration. The other workflows use Ecosystem B. Model assignments stay fixed while I vary assumed independent fallback coverage.
Of the four highest-criticality workflows (scores 4 and 5), Configuration One assigns fallback paths to three, leaving four of 18 criticality points uncovered. Configuration Two assigns a fallback only to customer remediation, leaving 14 of 18 uncovered. Yet both show 50% model concentration.
These are deliberately constructed assumptions, not operational test results. The comparison illustrates why concentration alone can be misleading: Two organizations with the same level of model concentration may have very different abilities to continue making important decisions when a system becomes unavailable or unreliable.
When A Model Update Changes Business Decisions
In April 2025, OpenAI rolled back a GPT-4o update in ChatGPT after the model became excessively agreeable. This was not an enterprise-wide failure; it illustrates how behavior can change even when a model remains available.
The practical question is how many important business decisions an update changes, not simply whether the model remains available.
First, use the dependency map to identify decisions exposed to the updated model. In the synthetic example, Ecosystem A supports procurement, financial approvals and maintenance planning. Exposure does not establish that any recommendation changed.
Next, business owners define what constitutes a material change and acceptable tolerance. Replay the same approved scenarios before and after the update, keeping inputs and controls fixed. Count changed business decisions, such as a supplier choice or purchase commitment crossing a defined threshold, not merely different wording. Report materially changed decisions divided by decisions tested, their criticality and which failed an independent check. Investigate other changes in data or configuration before attributing differences to the model.
Finally, connect the findings to action. If critical decisions remain within agreed tolerances and independent checks pass, stage the release and monitor. If a consequential decision changes unexpectedly or fails validation, stop automatic execution and investigate; revert the update, or route to a tested independent fallback. With no tested path, require business-owner review or pause the action.
This separates potentially exposed decisions from observed decision changes. The framework turns a behavioral signal into a release, escalation or recovery decision, rather than another dashboard alert.
Concentration measures where decisions are exposed. Change analysis measures what actually shifted. Validation and substitutability determine what leadership should do next.
Proving Progress In 90 Days
Track five measures: criticality-weighted model concentration, compute concentration, critical workflows without tested independent fallbacks, tested switching time and independent validation coverage. Record denominators and minimum acceptable service.
Start with consequential workflows, map dependencies and baseline both AI unavailable and AI available but wrong scenarios. After 90 days, evaluate whether recovery tests preserve acceptable business outcomes and whether validation controls catch material changes. Measure actual results rather than targeting an arbitrary improvement percentage.
The leadership question is not “How many AI vendors do we have?” but “Can we challenge or recover our important decisions when shared intelligence becomes unreliable?”
The objective is not model diversity. The objective is decision resilience.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?