# Visa's new test for AI shoppers rewards not buying > Visa's AI shopping benchmark passes a run that buys nothing when nothing fits. We built the gate between a shopping agent and the checkout. URL: https://www.voirlabs.io/insights/visa-ai-shopping-benchmark-not-buying Author: Moosa Jafri, AI agents & Web3, Voir Labs (https://www.voirlabs.io/team/moosa) Published: 2026-11-13 Tags: AI agents, Ecommerce, Automation If you're building a shopping agent, the agent is judged by the purchase. So when it doesn't know the budget it guesses, and when nothing in the store fits, it buys the nearest thing. Your customer sees a charge and then a return label. ## What Visa measured On November 5, Visa released what it calls the first industry benchmark for AI shopping. "The first round of tests involved 22 different AI models from 12 labs taking on 15 shopping challenges like buying a TV, laptop, or personal care appliance." Visa reports that models "failed 42.8% of test runs", and that "About 1-in-2 (54%) of failed runs still ended in the AI making a purchase." The rule that makes this interesting is in the same release: "If nothing fits the customer's requirements, the AI should opt not to buy." A run passes with the right purchase, or with no purchase when nothing fits. A payments company is rewarding a sale that never happens. TNW reported the same figures the same day. Some coverage ranked named models; Visa's own statement names none, so we don't either. ## A gate in front of the checkout We built the piece that sits between a shopping agent and the checkout, for a made-up electronics store on a sample domain. The model talks to the shopper, reads what they said and ranks products. Code keeps the list and opens the checkout. Four decisions carry it: - **The list is the store's, not the model's.** Each category has the merchant's checklist. A TV needs a budget, a size, the HDR format the shopper's videos or console use, and whether anyone games on it. The checkout stays shut until every line is known, and the missing line is the next question. - **A line counts only in the shopper's own words.** The model has to quote the shopper, and code finds the quote in the conversation. One shopper said "Something for the bedroom, around $400, 43 inches." The model added gaming, quoting "I play a bit". Nobody had said that, so the line was dropped and the gate asked. The answer was "No games." - **Every line is checked against the spec sheet**, never against the model's reading of it. A pick that misses a line is refused, and the next pick is tried. - **Nothing fits is a finished job.** If no product meets every line, the job ends with a note to the shopper, written by the model, and no charge. ## Twelve shoppers, with and without it Twelve sample shoppers went through it, against 40 product sheets, with the model's side of each talk scripted. The gate's code is real. The comparison is the same model without the gate: it buys its first pick from the shopper's first message, as an agent rewarded for finishing would. Without the gate, the agent bought 12 times, and 8 of those purchases were wrong, $4,932 charged in total. Through the gate it made 8 sales, none wrong, asked 14 questions, and sent 2 notes saying nothing fits. The case that turns is a shopper who opened with "My sister swears by Lumen". Without the gate, the agent bought a $1,099 Lumen set on that sentence. The gate asked four questions. The answers were "$1,000 tops", 65 inches, a camcorder that records in HDR10+, and a son who plays on his console. The $1,099 set doesn't show HDR10+. The gate bought the same brand's $949 set, which does. Another shopper wanted 75 inches, HDR10+ and 120 Hz for $600. Nothing in the store does all four. Without the gate, the agent bought a $599 set that does neither HDR10+ nor 120 Hz. Through the gate, the shopper got this: "Nothing at 75 inches does HDR10+ and 120 Hz for $600 or less. The closest is the Nadir 75 N6, which does both for $999. If the size or the budget can move, tell me and I'll look again." ## Our take The shopping agent's real job is finding out what someone needs. Buying is the last step, and it should be the easy one. Visa's benchmark scores it that way, and most agents aren't built that way yet, because a finished purchase is the metric everyone can see. If you keep one line: **nothing fits is an answer.** ## What asking costs Every line on the list is a question some shoppers won't wait for. Two of our twelve left before the last one. One wanted a hair dryer, answered two questions and walked away at the third; the agent without the gate would have sold them the right one, for $49. The other needed a laptop for school, and there the agent without the gate would have sold the wrong one. So the gate lost one good sale and stopped eight bad ones. That trade is its whole bet, and a store with a short checklist will make it better than one with a long one. The gate also knows only what the checklist asks: a need the merchant never wrote down, like a wall mount, is invisible to it. ## Three things to do before Black Friday 1. Pull last month's returns for anything your agent bought, and note which requirement each one missed. 2. Write the checklist for your top category as the merchant would, and make the checkout wait for it. 3. Write the note your agent sends when nothing fits, and count that as a finished job. If your agent spends a customer's money, we build the gate in front of the checkout. [Talk to us](https://calendly.com/voirlabs/30min) at voirlabs.io. ## Sources - Visa, "Visa Releases First-of-Its-Kind Industry Benchmark for AI Shopping; Almost Half of Test Runs Fail", November 5, 2026: https://investor.visa.com/news/news-details/2026/Visa-Releases-First-of-Its-Kind-Industry-Benchmark-for-AI-Shopping-Almost-Half-of-Test-Runs-Fail/default.aspx - TNW, "Visa tested 22 AI models as shoppers, and they failed 42.8% of the time", November 5, 2026: https://thenextweb.com/news/visa-ai-shopping-benchmark-agentic-commerce-ai-models-fail - The checkout gate, its README and the run: automations/agent-checkout-gate/ --- Voir Labs is an AI-first design and development studio (https://www.voirlabs.io/). Related service: https://www.voirlabs.io/services/ai-automation. Book a 30-minute conversation: https://calendly.com/voirlabs/30min