Mukul Arora drives Global Cloud Security, Privacy & AI Governance, GRC & Risk Management Consulting & Advisory at Persistent Systems Ltd.
Three years ago, I sat in a board meeting where a major enterprise proudly presented its new AI-powered fraud detection system. The slides were impressive, and the demo was flawless. The board approved the investment in under 40 minutes. But no one asked what data the model learned from. I’ve been in enough boardrooms since then to know that meeting wasn’t an exception. It was the rule.
The Questions That Matter Most
We’ve spent considerable energy teaching boards to ask about AI adoption: what it can do, how much it costs and what the competitive risk of not adopting it looks like. But most don’t ask the questions that matter most: Can we trust what this model has learned, and how can we confirm the model’s learnings are correct?
This points to one of the most consequential and least understood risks in enterprise AI today: model poisoning, or the deliberate corruption of the data an AI system learns from.
What Model Poisoning Looks Like
Model poisoning doesn’t announce itself. It doesn’t trigger your SIEM. It doesn’t show up in your threat intelligence feed. It simply teaches your AI to be wrong consistently, invisibly and at scale.
Consider what that means in practice. Your AI-powered fraud detection system, trained on subtly manipulated data, begins approving the very transactions it was built to block. Your cybersecurity AI classifies active malware as safe because its training set was compromised months before deployment. Your AI hiring tool systematically filters out the right candidates because someone poisoned its learning data upstream. By the time the damage surfaces, the decisions have already been made, the transactions already processed and the risks already materialized.
This is what makes model poisoning uniquely dangerous. It exploits the time between when AI systems are built and when their failures are discovered.
The Governance Problem
Most organizations heavily invest in securing AI outputs: the responses, recommendations and decisions the model produces. Very few secure AI inputs: the data pipelines, training sets and model supply chains that determine what the AI believes to be true.
I’ve advised boards across BFSI, healthcare and manufacturing on cybersecurity and AI governance, and I consistently see organizations treating AI model integrity as a technical problem owned by the data science team. It isn’t. It’s a governance and risk management problem, as well as a fiduciary responsibility. When a board approves an AI system without asking about training data provenance, adversarial testing or model supply chain integrity, it isn’t being negligent about technology. It’s being negligent about governance.
Three Things Every Board Should Demand Before Approving an AI Deployment
The good news is that solving this governance problem doesn’t require boards to become data scientists. It only requires them to ask three questions:
1. Where did this model learn what it knows and can we verify the integrity of that data? Training data isn’t a technical detail. It’s the foundation of every decision the model will ever make. Boards should demand data provenance frameworks that track the origin, integrity and chain of custody of every dataset used to train or fine-tune AI systems. If the answer to this question is vague, you know not to move forward with that model.
2. Has this model been tested for manipulation, not just performance? Standard AI testing measures accuracy, speed and reliability. Adversarial testing asks: If someone wanted this model to fail in a specific way, how would they do it? Red-teaming AI systems before deployment is the minimum standard of responsible governance, especially in high-stakes environments.
3. Who’s accountable when this model is wrong, and does that accountability have teeth? Accountability without consequence isn’t accountability. It’s a disclaimer. Boards need to establish clear ownership of AI model integrity, not just at deployment but continuously, as models are updated, retrained and exposed to new data environments.
The Conviction That Should Drive This
I’ve spent over two decades at the intersection of cybersecurity, governance and enterprise risk. The conviction that’s guided every chapter of that career is simple: Security isn’t a constraint. It’s a business enabler. The same is true for AI governance.
Organizations that build trustworthy AI, not just capable AI, will earn the confidence of customers, regulators and boards that their competitors can’t match. The board that approved that fraud detection system three years ago wasn’t wrong to be excited about AI. They were wrong to assume that a vendor demo was sufficient evidence of trustworthiness.
The next AI failure in your organization may not begin with someone breaking in. It may begin with something already inside being permitted to do too much, with data nobody verified, approved by a board that never thought to ask.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?