Fernando Fainzilber is the CISO at Start Campus, leading security for AI-scale sustainable data center infrastructure in Portugal.
I repeatedly hear a version of the same question: What happens if an AI system goes beyond its guardrails and turns on humans? It sounds like a Terminator movie scene, which is partly why it’s asked so much.
While it’s a fair question, there’s a more relevant problem I already see happening often: people handing over their judgment to AI, one “easier decision” at a time.
It makes sense, since the brain expends a lot of energy and is, therefore, programmed to economize accordingly. Daniel Kahneman writes about this in Thinking, Fast and Slow as the “law of least effort.” When offered two ways to the same goal, people tend to choose the one that costs less mental effort. Intentional reasoning (Kahneman’s System 2) is recruited only when something forces it. Any tool that supplies a decent answer without System 2 wins by default.
AI is the best tool we have ever built for removing effort, but leaders will need to ensure their teams don’t lose the skills needed to do their jobs in the process of using it.
The Impact Of Overreliance On Automation
The effect of offloading decisions to AI is now measured.
In a 2025 study presented at the CHI conference, researchers at Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers across 936 real work tasks. Two findings from the survey deserve our attention:
1. Higher confidence in generative AI predicted less critical thinking about the output.
2. The shape of the work changed, from producing answers to checking them, and from performing tasks to supervising them. In this case, supervision only counts as a control if the supervisor can tell when the machine is wrong.
In my industry, cybersecurity, these findings have high stakes.
Security operations centers (SOCs) have been automating triage for years, for obvious reasons. Constantly increasing alert volumes have exceeded human capacity, leading to analyst fatigue. However, automation bias and complacency in security operations are also major concerns, with humans becoming over-reliant on tools and no longer looking around the corners for contradictory evidence.
Lisanne Bainbridge called this unconscious trap an “irony of automation“ decades ago. Automating the routine cases strips away the repetitions that built humans’ judgment. Then you hand that human the exception. In the security field, the exceptions are where breaches live, and where trained judgment is mostly needed.
The public version of this was well demonstrated in a Canadian legislature in June 2026, when a politician read AI editing instructions aloud during a speech. The error became a joke for a week, and most coverage stopped there. The most important detail is that nobody in the chamber reacted. A room full of professional listeners let machine-written text pass without people noticing or triggering a verification instinct.
How To Challenge AI Outputs
Months ago, a colleague from another company called me with a concern. His team members were leaning more and more on AI tools to keep pace with increasing work volume caused by adversaries themselves driving up attacks with AI. He worried his analysts were becoming used to handling the tools to accelerate their response without actually thinking about the underlying problems.
I suggested one question, borrowed from Simon Sinek’s Start With Why: Whenever an analyst suggests something, ask “why” that was the recommendation and what would be the other options.
He came back surprised, as a big share of the answers were generic, like “industry best practices,” “it made sense in the circumstances” and, most honestly, “the tool recommended it.” His people had not stopped working, but they had stopped reasoning. No one had noticed, because the output still looked right.
The usual delegation decision to automate the repetitive and reserve the creative is not enough. Instead, I use the underlying cost criteria. It works by asking three questions whenever using an AI tool:
The first: If the output is wrong, will we notice it? Grammar correction and spreadsheet population errors are easy to notice, cheap to correct and usually cheap even if not corrected. However, a risk assessment, a vendor evaluation or a root-cause narrative are hard to notice and usually expensive to fix. Detectability and cost, rather than creativity, should decide what goes to AI.
The second: What practice and on-the-job training are we removing? If professionals never do manual boring work like triaging routine alerts, they will never develop the pattern recognition demanded by the hard alerts. Preserving some manual, inefficient repetitions is sometimes a training cost that can’t and shouldn’t be removed.
The third: Who owns the decision and the responsibility? A person must be able to explain the reasoning, not the tool. “The system flagged it” will never be enough for a regulator, a board or an incident review panel.
Thinking Alongside AI
I don’t think a modern security function can be efficient and effective without leveraging AI. AI genuinely returns the single scarce resource we cannot manufacture: time.
The question is what we spend that time on. If it goes to better analysis and questions, we win. If it goes to more throughput and shallower thinking, we have traded a durable capability for a temporary metric. Poor choice, usually unnoticed.
Whether AI will take over your organization should be a secondary consideration. The more important concern now is whether the people in the organization are practicing the judgment you will need from them on the day the tool is confidently wrong.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?