Rajesh Iyengar, Founder & CEO of Lincode Labs and author of Teach the Factory to See.
When manufacturing leaders evaluate AI’s inspection capabilities, the conversation often begins with a percentage: How accurate is the model? It is a reasonable question. But in my experience developing and deploying visual inspection systems, it is rarely the most useful starting point.
I prefer to ask a different question: What happens when the inspection decision is wrong?
A defective component may move into another operation, accumulating material and labor costs before anyone discovers the problem. An acceptable component rejected unnecessarily may trigger reinspection, delay production or consume an engineer’s attention. Both errors matter, but their consequences differ.
Factory trust begins when leaders understand those consequences and design the inspection system around them.
A strong accuracy score can hide a weak inspection process.
Consider a hypothetical evaluation containing 990 acceptable parts and 10 defective ones. A system that passes every part achieves 99% overall accuracy while missing every defect.
The arithmetic is correct, but the inspection is useless.
Overall accuracy can mislead when one class is rare or different mistakes carry different costs. In Chapter 13 of my book, Teach the Factory to See, I explore this distinction, but my main takeaway is as follows: A reassuring score can conceal an error profile the factory cannot afford.
Leaders should ask what sits behind the number. Which defects were tested? Did the evaluation include borderline cases and acceptable variation? Were reference decisions agreed upon by qualified quality personnel?
Track missed defects and false rejections separately, using clear denominators. Examine performance by defect type and severity. Connect those measures to the operating problem the project is supposed to solve.
An impressive average should never conceal poor performance on the defect that justified the investment.
Test the conditions the factory actually produces.
In one manufacturing application documented by our engineering team, cylinder heads were inspected before washing. Their surfaces could carry machining oil, coolant residue, dirt and loose metal shavings. Reflections further complicated the image.
Those conditions were part of production. Treating them as unusual exceptions would have meant designing for a factory that did not exist.
Leaders need to evaluate inspection under the conditions the production process actually creates.
The challenge extends beyond the model. Camera selection, optics, illumination, computing and inspection logic all have to fit the component condition and quality requirement.
As such, before accepting a pilot, require evidence from representative conditions: different shifts, material variation, part presentation, surface finishes and expected contamination.
Confirm that the sensing method can measure the required characteristic. If acceptance depends on scratch depth, detecting a visible scratch in a conventional two-dimensional image does not establish that its depth meets the specification.
A better algorithm cannot compensate for missing measurement information.
Resolve the quality standard before automating it.
A model cannot settle a disagreement between engineering, quality and production about what constitutes an acceptable part.
Address that disagreement before training and evaluation.
I recommend building a shared collection of examples covering clear defects, acceptable variation and uncertain cases. Record the disposition and reasoning. Identify who can approve changes to the standard.
Operators belong in this process. They see recurring conditions, handling differences and shift-specific problems that may never appear in a carefully prepared demonstration.
Their involvement creates a practical route for reporting mistakes. A recurring false rejection should become evidence for investigation rather than an inconvenience people quietly work around.
Make the inspection result operational.
Correctly identifying a defect is only one part of a functioning inspection process.
The decision must arrive in time, reach the appropriate equipment or person and trigger the intended action. An uncertain result needs an agreed path for review. Someone must own the final disposition.
Before deployment, ask who can stop the line, authorize reinspection, override a result and approve a configuration change. Define what happens when the system becomes unavailable. Preserve enough evidence to reconstruct a disputed decision, including the relevant image, inspection result and configuration version where appropriate. These details can determine whether the technology becomes useful during a difficult shift.
Treat deployment as the beginning of accountability.
Production changes after installation. Materials, tooling, lighting and processes evolve. An inspection system needs monitoring and controlled updates throughout its operating life. Post-deployment monitoring is crucial for assessing AI reliability under real-world conditions.
To address this, leaders must assign responsibility for reviewing errors, investigating changes and validating updates before wider release.
The leadership question is broader than “How accurate is our AI?” Can the factory explain its decisions, act on them consistently and recognize when performance changes? This is the evidence manufacturing teams need before they can place their trust in AI inspection.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?