Guide
What is AI search monitoring? Metrics, methods, and limits
AI search monitoring measures whether AI assistants mention, cite, and recommend your brand. Here is what it measures, how it works, and where it stops.
Short answer
AI search monitoring is the practice of repeatedly asking a fixed set of buyer questions across AI answer surfaces and recording whether a brand is named in the answer, cited as a source, how early it appears, how it is characterized, and how much of the answer space it holds against competitors. It measures visibility inside generated answers, which ordinary rank tracking and web analytics cannot see.
Buyers now ask an assistant before they open a search results page. When they do, one answer is returned, a handful of brands are named inside it, and the rest of the market is not shown at all. AI search monitoring is how you find out which side of that line you are on.
This guide covers what the category measures, how the measurement works, what it cannot tell you, and how to evaluate a tool that claims to do it.
The five signals worth separating
Most confusion in this category comes from collapsing distinct signals into one “visibility score.” They answer different questions and they move independently.
| Signal | Question it answers | How it is detected |
|---|---|---|
| Mention | Was the brand named in the answer? | The brand name or a configured alias appears in the visible answer text |
| Citation | Was the brand’s site used as a source? | A domain the brand owns appears in the answer’s source URLs |
| Position | How early was the brand named? | Order of first mention among the tracked entity set |
| Sentiment | How was the brand characterized? | Classification of the answer’s language about that brand |
| Share of voice | How much of the answer space did the brand hold? | The brand’s mentions as a proportion of all tracked entities’ mentions |
A brand can be mentioned without being cited, cited without being mentioned, both, or neither. Treating those four states as one number throws away the part that tells you what to do next.
What AI search monitoring is not
It is not rank tracking. Rank tracking reports a position in an ordered list of links. An AI answer has no ordered list. The useful question is inclusion, not position, and the same query can return a different answer minutes later.
It is not social listening. Brand monitoring tools watch what people publish about you. AI search monitoring watches what a model says about you when a buyer asks it a question. The inputs overlap. The measurement does not.
It is not an optimization service. Measurement tells you where you stand and what changed. Changing the outcome is content, PR, product, and positioning work. Be skeptical of any tool that reports the number and sells the fix without separating the two.
How the measurement actually works
Four stages, in order. Each one has a failure mode worth knowing about.
1. Freeze the question and entity set
A monitored workspace holds one brand, its competitors, the aliases each entity answers to, the domains each entity owns, and the buyer questions to ask. Every run should freeze that set at the moment it starts. If a competitor is added halfway through collection, the finished run must still use the snapshot it began with, or the answers inside a single run stop being comparable.
Aliases are where false positives get created. A brand whose name is also an ordinary English word needs case-sensitive matching, or every occurrence of that word in an unrelated sentence becomes a mention.
2. Collect repeatedly, on a schedule
AI answers are not deterministic. The same prompt on the same surface can return different text, different sources, and a different set of named brands from one run to the next. That is a property of the systems being measured, not a collection bug.
The consequence is structural: a trend built from completed runs over time is evidence, and a single answer is an anecdote. A tool that shows you one run and calls it a ranking is reporting noise as signal.
3. Score the visible answer
Scoring runs over two things: the answer text a person would read, and the source URLs attached to it. Two details matter more than they sound like they should.
Provider response shapes drift. A scorer that expects one fixed schema silently returns zeros when a provider changes a field name, and zeros look exactly like “you were not mentioned.” A scorer that walks the whole response degrades to missing nothing instead.
Source URLs need cleaning before they count as citations. Some surfaces put a full URL in a citation’s domain field, which produces a bare site root that duplicates a deeper link from the same origin. Favicon and thumbnail assets arrive in icon fields. Neither is a citation, and counting them inflates the number that a buyer is most likely to check.
4. Aggregate with explicit denominators
Rates need stated denominators. “42% mention rate” means nothing until you know it is 42% of which prompts, on which surfaces, over which runs. Comparisons need a shared basis too: if run A covered thirty prompt and surface combinations and run B covered eight, only the eight they share can honestly be compared. Without that rule, a partial run manufactures a dramatic change that never happened.
What it cannot tell you
State these limits before a stakeholder finds them.
- It does not measure traffic or revenue. AI answers frequently resolve the question without a click. Visibility inside the answer is the outcome being measured, and attributing pipeline to it requires separate evidence.
- It measures the prompt set you chose. A well-built set of buyer questions represents a market. It does not enumerate it. Any claim that a tool measures “all AI search” is a claim about a sample presented as a census.
- It cannot reproduce every user’s answer. Geography, session history, personalization, and product changes all move the result. What a monitor records is what was observed at that time, from that collection context.
- A missing Google AI Overview is a real observation. AI Overviews appear on a minority of queries. When one does not appear, the correct record is that no overview was shown, not that collection failed. Tools that discard the empty case overstate how often you were absent from something that never ran.
How to evaluate a tool in this category
Six questions, in the order that separates the field fastest.
- Can I see the raw answer behind any number? If a metric cannot be traced to the text it came from, it cannot be checked, and an unverifiable number fails the first time leadership questions it.
- Are mentions and citations reported separately? A single blended score hides the distinction that determines what you do next.
- How does it handle non-determinism? Ask whether the product compares completed runs over time or presents one answer as a ranking.
- What is the denominator? Every rate in the interface should say what it is a rate of.
- Which surfaces, and are they separated? Google AI Overviews and Google AI Mode are different products with different behavior. Merging them into “Google” loses the distinction.
- Can the calculation be audited? Published methodology, and ideally the source itself, is the difference between a measurement and a claim.
Where refd sits
refd is a measurement tool. It runs a fixed prompt set across ChatGPT, Perplexity, Gemini, Google AI Mode, and Google AI Overviews on a schedule, records mentions, citations, first-mention position, sentiment, and share of voice as separate signals, and keeps the raw answer behind every metric so any number can be inspected rather than trusted. The scoring contract is published, the stack is MIT licensed, and it can be self-hosted if you would rather hold the raw data yourself.
It does not write your content and it does not promise to move the numbers. It tells you what the numbers are, and shows you the evidence.
See your evidence