Methodology

How every number is made

Before you put an AI visibility number in front of your CMO, you should know where it comes from. This is the whole method: the one-off check, weekly tracking, the three metrics, and how findings are ranked. Nothing hidden, nothing hand-tuned.

1. Writing five buyer questions

You enter a brand and a category, such as “CRM for B2B sales teams”. Mentioned asks Claude to write 5 questions a buyer would type when looking for a recommendation in that category, phrased the way a person asks, not the way a marketer would. The questions never contain your brand name, so the test measures an unprompted recommendation rather than a leading one. If question writing fails for any reason, Mentioned falls back to a fixed set of neutral templates rather than guessing or skipping the check.

2. Asking each assistant, independently

All 5 questions are sent to each selected model in parallel, each as its own conversation. There’s no shared context between them and no hint about which brand is being tested, so it’s close to five different buyers asking five different questions. Claude, ChatGPT and Gemini run on our keys. DeepSeek, Llama 3.3 and Perplexity run on an API key you supply.

3. Counting a mention

The matcher reduces both your brand name and the answer to letters and digits, so spacing and punctuation can’t break a match, and it folds accents, so “Nestlé” and “Nestle” both count. It then requires the match to land on a word boundary in the original text. Without that rule, “generally” would count as a mention of a brand called “Rally”.

A brand whose alphanumeric core is under two characters (“C++”) is matched literally instead. A bare single letter with nothing to anchor on (“X”, “R”) is declined rather than guessed at: there is no reliable way to tell it from the pronoun.

When there is a match, Mentioned keeps the sentence that contains it as the quote you see in your results, copied straight from the model’s output and never rewritten. Highlighting on stored answers uses the same matcher that scored the run, so a highlight can never contradict a score.

4. Naming who got recommended instead

When your brand doesn’t appear, Mentioned pulls the other brand names out of the answer, usually bolded names or items in a numbered or bulleted list, and filters out generic phrases so “the best option” isn’t counted as a competitor. It’s a heuristic and can miss a name, but it never adds a competitor that isn’t in the text.

5. The score

Your score is how many of the 5 answers mentioned your brand, out of 5, for each model. There’s no weighting and no adjustment. 3/5 means your brand showed up in 3 of the 5 independent answers.

6. Locking the questions, so a trend means something

A one-off check writes fresh questions every time. That’s right for a snapshot and wrong for a trend: if the questions change from one Monday to the next, a score moving from 3/5 to 1/5 mixes “we lost visibility” with “we asked harder questions”.

So when you track a keyword, its five questions are written once, when you create it, and asked word for word on every run after that. A move in the line is a move in your visibility. It’s the most important design decision in the product.

Tracked keywords run weekly on our key, on Claude and Gemini. Pro includes 3 daily slots on our key and Agency 10. Beyond those, a keyword can run daily only when your account has saved its own API key for every model that keyword asks, because that daily run spends your key rather than ours. The check happens when you set the schedule and again when the run starts, so removing a key later moves the keyword back to weekly.

7. The three metrics, and why you need all three

Each one counts something a model actually returned, and every row links to the full answers it came from. Where a figure couldn’t be measured you get a dash and a sentence saying why, never a zero. “Nobody named you” and “nothing has run yet” are different findings, and Mentioned never shows one as the other.

8. Reading the tone of the sentence that names you

Sentiment is a word count, not a model’s opinion. Mentioned takes the sentence that names your brand, counts listed praise and criticism words in it (handling negation), and calls the sentence positive, negative or neutral by which side has more. The full word list is on the screen, with how often each word fired and the sentence it fired on, so you can check any verdict in one click.

It will miss sarcasm, and it will miss praise phrased in words that aren’t on the list. Showing the whole list is the trade. A sentiment score that a model was asked to invent couldn’t be checked this way at all.

9. Ranking what to fix

The Actions screen reads the runs already on your account and lists eight kinds of finding: model blind spots, prompt blind spots, visibility drops, competitors pulling ahead, a competitor quoted where you aren’t, weak topics, keywords that never ran, and keywords whose last run failed. Each finding is ranked by the answers at stake: the gap in percentage points times the number of stored answers it covers. Every row restates a measurement that’s already on another screen and links straight to it, and the list tells you how many findings it left out rather than cutting them silently.

Under each finding, a collapsed playbook gives the usual fixes for that kind of gap. It’s labelled as advice, and it never enters a number, the CSV, the API or the MCP tools.

10. What a run costs, and what happens when one fails

Every check spends real API budget. That’s why there’s no anonymous check, why a free account gets 3 checks on our key before it needs a key of its own, and why there’s a 300-a-day global cap above everything. Weekly runs are metered in real token counts against a monthly dollar budget. A keyword the budget can’t cover this month is deferred, not failed: nothing is recorded against it, because a red mark on a healthy keyword would be wrong.

A request that fails in a way that can clear on its own (a rate limit, a timeout, a provider outage) is retried, following the provider’s own retry hint. A rejected key or a malformed request isn’t retried, because a retry can’t fix it. A failed run is recorded rather than hidden, and the schedule reads the last successful run, so one timed-out request doesn’t leave a keyword idle for a week.

11. From numbers to fixes

The method exists so your team can act on the numbers with confidence. In practice the loop looks like this:

  1. Start with Actions. The top finding is the gap costing you the most answers. Open it to read the answers behind it.
  2. Use the playbook as a starting point. It lists the usual fixes for that kind of gap, such as publishing the specific claim a competitor gets credit for, or getting covered on the sites an assistant cites.
  3. Ship the change, then watch the same question. The questions are locked, so a move after your change is a real move rather than a reworded prompt.
  4. Check whether it turned into visits. Connect Search Console and GA4 to see your search queries and the sessions that came from AI assistants, next to your AI visibility. The AI gap report goes one step further: it asks the queries you already win on Google to the assistants, with your say-so, and shows which ones never name you.

What Mentioned does not do

Run a free check See the screens →