Does ChatGPT give the same answers to everyone? Why answers change and what it means for your brand

No. ChatGPT does not give the same answers to everyone, and often not even to the same person twice. Ask it for the best project management tools, open a new chat, ask again, and the list may change: different brands, a different order, a different length. Ask from another account and it can change again.

For most users this hardly matters. For a brand that wants to know whether ChatGPT recommends it, it changes how you have to measure. This guide explains the six reasons the answers differ, how large the differences are, and how to get reliable numbers anyway.

Why ChatGPT gives different answers to the same question

Six things change the answer. Some change it from one run to the next for the same person; others change it from one person to another.

1. The model picks its words with some randomness

A language model writes one word at a time, choosing each from a list of likely words. It does not always take the most likely one: a setting called temperature controls how much variety it allows, and chat assistants keep some variety so answers don't sound mechanical. A different word early on leads to a different sentence, and sometimes to a different brand.

Even with that randomness turned off, answers are not fully repeatable. Researchers at Thinking Machines Lab showed in 2025 that the main cause is the server load: requests are processed in batches whose size changes with traffic, and that changes the arithmetic just enough to change the output.

2. Web search finds different pages

When a question needs current information, ChatGPT searches the web. According to OpenAI's help center, it rewrites the question into targeted queries sent to its search providers, and may send follow-up queries after reading the first results. Search results move, the queries it writes can differ from one run to the next, and the answer follows the pages it found. This is why cited sources are worth tracking on their own: our guide to where ChatGPT gets its information explains the two ways it answers.

3. Memory and past chats

ChatGPT can remember details you have shared and refer to past conversations. OpenAI says memory makes answers more relevant and personalized, and that it can also shape the web searches ChatGPT runs for you. Someone who has told ChatGPT they run a small agency gets different software recommendations from someone who works in a large company, even with the same question.

This matters for tracking. An account that has asked the same 20 questions about furniture every week for months is not a neutral reader any more.

4. Location and language

ChatGPT shares a general location with its search providers, and local questions depend on it: "best family lawyer near me" has no single answer. The language of the question changes the sources too. An Italian question gets Italian sources and often Italian brands.

5. The model and its version

Free and paid plans, and the model chosen in the picker, can give different answers. Models are also updated over time, so the same account can get a different answer in March than in January. Keep a note of the model you track, and expect a step change when a new one ships.

6. Wording and the conversation so far

Small changes in wording change the answer, and so does everything said earlier in the same chat. People who ask about the same need rarely use the same words, which is why our guide on how to choose prompts to track AI visibility starts from intents rather than exact phrases.

How different are the answers, really?

Very different in the details, surprisingly stable in substance. SparkToro and Gumshoe ran a dozen brand recommendation prompts thousands of times on ChatGPT, Claude and Google's AI at the end of 2025. As reported by Search Engine Journal, ChatGPT and Google returned the same list of brands less than 1% of the time, and the same list in the same order less than 0.1% of the time. Claude repeated its lists a little more often but changed the order more.

Yet the leading brands kept coming back. For headphones, four brands appeared in 55% to 77% of almost a thousand answers, even though the prompts were written by different people. The authors' conclusion: position in an answer means little; how often a brand appears across many answers is the number that holds up.

The other assistants work the same way. Gemini, Perplexity, Claude and Google AI Mode all generate their answers and most of them search the web, so all of them vary. Google itself says its AI Overviews can make mistakes, and suggests rephrasing a question or asking for other viewpoints.

What this means for measuring brand visibility

If every answer is a draw, a single answer is a sample of one. Three consequences follow.

Count appearances, not positions. The visibility rate (the share of answers that name the brand) is stable across many answers. The position in a single answer is not, and should not be reported.

Measure over many answers. Twenty questions on five platforms give 100 answers per run; weekly runs give about 400 a month. That is enough to see real changes over a few months. Our guide on whether you can track brand mentions in AI search shows how wide the margin of error is at each size.

Read trends, not snapshots. A brand that goes from 22% to 34% over a quarter has moved. A brand that goes from 30% to 33% in a week has not, necessarily. The AI visibility score page explains the metrics in detail.

How to set up accounts for tracking

Whatever tool you use, the accounts that ask the questions decide what you measure. A few settings keep them close to a new visitor.

Use accounts reserved for tracking. Don't track from the account you use every day for work. A separate Chrome profile with its own logins keeps the tracking accounts clean.

Turn memory off. In ChatGPT, open Settings and turn off memory (depending on the account, one Memory switch or separate switches for saved memory and chat history). Delete anything already saved. Leave custom instructions empty.

Keep the country and language fixed. Run from the same country every time, in the language of the market you track.

Keep the model the same, and note when the platform changes it.

For a quick check by hand, use a Temporary Chat: OpenAI says it neither uses nor updates memory. Ask the question in a new temporary chat three to five times and count how often the brand appears.

Measuring visibility with QuotedBy

QuotedBy runs your questions in your own Chrome on ChatGPT, Claude, Gemini, Perplexity and Google AI Mode, so the answers are the ones the real interfaces give, with your accounts and your country. It types each question, you press Enter (Google AI Mode starts on its own), and it saves the answer with its sources and a screenshot.

Each run asks every question once on each platform. The numbers QuotedBy shows are rates over all those answers, and the comparison with the previous run tells you which questions and platforms you gained or lost. Reminders (daily, weekly or monthly) keep the runs regular, which is what turns single answers into a trend. Installing QuotedBy in a Chrome profile reserved for tracking, with memory off, follows the setup above.

The free plan covers one project, three competitors and 20 questions on all five platforms. Pro, at 24 € a month, raises the limits to 10 projects and 80 questions per project.

Frequently asked questions