Who checks the AI tools you use, and what should you ask?
Picture Nell, who runs a specialty food company and has just sat through a demo of an AI assistant for her online shop. It was impressive. It answered every question about her own jams correctly, which is the easiest test there is. What she really wants to know is who checked the thing, and what to ask before she signs.
This is part two of AI for Good, the Yield & Press series on using artificial intelligence (AI) in ways you would be happy to explain to a customer.
Who tests AI models before they are released?
Three kinds of people, and the good news is that they exist at all.
The companies that build the large models publish documents, usually called system cards or model cards, that describe how a model was tested and what it got wrong. Independent evaluators test too. METR, which describes itself as “a research nonprofit that evaluates frontier AI models to inform the public about their risks and capabilities,” runs evaluations before some models are released.
Governments have joined in. The United Kingdom’s institute, founded as the AI Safety Institute, was renamed the AI Security Institute in 2025. The US equivalent became the Center for AI Standards and Innovation the same year, and the international network they belong to was renamed the International Network for Advanced AI Measurement, Evaluation and Science. The word “safety” has been leaving the letterheads. The institutes still test models.
What does that testing tell a small business?
Less than you would hope. Those evaluations test the model underneath, for things like whether it can be talked into helping with something dangerous. Nell is not buying the model. She is buying a product somebody built on top of it, with its own instructions, her data and her customers.
Whether that assistant tells a shopper the jam is nut-free when it is not, or keeps a copy of every conversation for three years, is a question about the product. Nobody at an institute is going to answer it for her.
What should you ask an AI vendor before you sign?
- Do you train on my data, or my customers’? Get the answer in writing. For comparison, OpenAI’s enterprise privacy page says, of its business products and its API: “By default, we do not use your business data for training our models.” A vendor should be able to point to a sentence like that.
- Which model is underneath, and does its maker publish a system card? A vendor who will not say which model it uses is asking for trust it has not earned.
- Will it tell my customers they are talking to AI? In the EU, the AI Act has required chatbots to be built to disclose this since 2 August 2026, and a customer who finds out later is rarely pleased.
- What happens when it is wrong? Who fixes it, how fast, and can you read the conversations? You answer for what it says on your site.
- Can a customer reach a person? There should always be a way out of the chat to somebody who can make a decision.
- Where is the data kept, and for how long? Conversation logs are customer data. Treat them like the rest.
What is a red flag in an AI sales pitch?
A promise of 100% accuracy is the loudest one. The major model makers publish system cards describing known limitations and errors, so a reseller promising perfection is promising more than the maker does. Close behind: vague answers about data, and no named model at all.
A vendor who says “we don’t know yet, let me find out” is fine. A vendor who says it cannot possibly make a mistake has just made one.
Where does this leave Nell?
With six answers in writing and a much shorter list of vendors. The one she picks introduces itself as the shop’s assistant, hands anything about allergies to a person, and keeps conversation logs for ninety days. Not the flashiest demo. The one she can explain to a customer.
Common questions
Who tests AI models before they are released?
Three groups. The companies that build the models publish system or model cards describing their testing; independent evaluators such as METR evaluate some models before release; and government institutes such as the UK AI Security Institute and the US Center for AI Standards and Innovation test models as well.
Does independent AI testing cover the product I am buying?
Usually not. Those evaluations test the underlying model. A product built on top of it adds its own instructions and your data, so questions about accuracy, data use and disclosure have to be asked of the vendor.
What should I ask an AI vendor about my data?
Ask whether they train on your data or your customers’ data, where it is stored and for how long, and get the answers in writing. OpenAI, for example, states that by default it does not use business data from its business products for training.
What is a red flag when buying an AI tool?
A promise of 100% accuracy, vague answers about data, or a vendor that will not name the model underneath.
Want to know what AI says about you?
The AI Visibility Check asks ChatGPT, Gemini and Perplexity about your business and shows you what comes back.
Run the free check →How Not to Be Invisible Online
Essential web, SEO, and digital strategy alerts built for independent business owners who want to get found online.
Respecting your inbox. Unsubscribe anytime. See our Privacy Policy.Editorial. Sources: METR (metr.org); OpenAI Enterprise Privacy; public announcements of the UK AI Security Institute and the US Center for AI Standards and Innovation. DigiVino, September 2026.