Srijith Ravikumar is a Principal Engineer at Amazon building AI-powered recommendation systems at scale. Published researcher at AAAI.
When a customer photographs a sweater and asks what it is, an AI system has to decide what it’s looking at. That answer might come from your product data or from a classifier that never consulted you.
Google puts Google Lens at more than 25 billion queries per month, with one in five showing commercial intent.
Your Product Data Controls The Match
To understand why your product data matters, it helps to look at what happens between the customer’s camera and the products the system ultimately returns.
Camera search does not compare the shopper’s picture to yours pixel for pixel. In a 2020 paper, Pinterest’s engineers described their production system as extracting a visual embedding, a numerical fingerprint of how the object looks, and fetching its nearest neighbors from an index of product images. That means your catalog data is not necessarily what identifies the initial visual match.
The same paper explains why its engineers could not stop there. Retrieving on visual similarity alone, they write: “may result in large semantic mismatch errors, such as returning candidates from a different category than the query object.”
The fix was to make category a restriction on retrieval itself, and as the system grew, other attributes joined it in queries like “gender:Men AND (category:Shirt OR category:Tie) AND (NOT price < 50)”.
Your attributes are not what finds the product. They are what decides which candidates the system is allowed to retrieve at all.
AI Makes Classification Someone Else’s Decision
Google now describes image search as Gemini deciding which tools to call, with the model acting as the brain and visual search as the library. My read: As models take on more of the reasoning, your product data becomes less of a direct anchor, and that makes the quality of the attributes feeding those systems more consequential.
Then the sentence that should change how you staff attribute quality. Pinterest groups candidates by category, in the engineers’ words, using “either an in-house Google Product Taxonomy (GPT) classifier or merchant-provided GPT labels.” Read that acronym as Google’s product-category tree, not the language model.
“Either.” Their classifier and your labels occupy the same slot.
Google goes further in its own help pages: All products are automatically assigned a category from its taxonomy, and a merchant value is accepted as an override only in specific cases. Feed teams have known this about Google Shopping for years. The new part is that it now runs behind a camera. When your data is missing, stale or bucketed badly, you have not opted out of classification. You have left it to a stranger’s inference about your own inventory.
Merchant Center sends you a report when a feed throws an error. It does not send you one when your product-type value is replaced. You get error reports. You do not get override reports.
Treat Attributes As A Source Of Truth
Start with your product-type field. Look at how the neighboring fields get filled, starting with color. In the catalogs I have worked on, a coordinator picks from a short list at style setup, working from a swatch or the product name, once, and nobody revisits it. Twenty buckets is right for a shop-by-color filter, so this is not a failure of merchandising judgment. The failure is that the coarse bucket is the only color value in the building.
There is often a better number two system upstream. I build search and recommendation systems, and the pattern I see most is not a bad model but a field nobody has examined since setup. Apparel brands already measure color precisely in development: the dye house has to hit a numeric tolerance, so the first swatch is read on an instrument that reports color as numbers. The Google Merchant Center product data specification has no field for it, so treat it as a referee for disagreements rather than something to pipe downstream.
The sharpest case is wholesale. Your vendor supplies the same photograph to 10 retailers. No embedding can separate you from any of them, because the pixels are identical. What distinguishes your listing is attributes, price and whatever authority the channel assigns you. Your labels are the entire tiebreak, and it is not a comparison most brands run.
Measure Whether AI Sees What You Sell
Merchandising owns the words. The studio owns the images. Product data owns the feed going to Google, the marketplaces and the ad platforms, often under a manager whose mandate is that feeds do not throw errors. Attribute quality sits between them. The owner exists, pointed at the wrong number.
Give that person one number: the share of your catalog where attributes agree with the photograph.
Start with data you have. Your returns system probably holds free-text comments saying the item was not what the customer expected, tied to SKU and image. Run that first. Then write a scoring rubric before anyone photographs anything, and take 25 products: bestsellers, plain basics, prints and a few where you suspect the name is odd. Photograph each in three conditions a customer would actually produce: poor light, an angle, a sofa. Run them through a general visual search, not your own site search. You are measuring whether the system agrees with your catalog about what the thing is.
Look for three patterns: whether prints behave differently from plain items, whether plain items return a wall of competitors, and whether your products surface under a retail partner’s listing. The last is worth escalating.
Skip the vision vendor for now. Buying before you know whether your attributes agree with your photographs solves a problem you have not diagnosed. If you do buy, the audit output is your acceptance criteria.
Rank the payoff by what you can attribute. Start with feed rejections. Google requires color on apparel-free listings, and on apparel Shopping ads in six countries, including the U.S. and U.K., and can disapprove products that are missing it. The channel reports those failures, and a suppressed item earns nothing. Next, look at your shop-by-color filter, where complaints and usage are testable on your own site. Color-mismatch returns may be the largest problem, but they are harder to attribute directly to product data. Start with the first two, where the connection between the data and the business impact is easier to demonstrate.
Your customer holds up a phone and asks what something is. There’s an answer. The only question is whether it is quoting you.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?