Track AI visibility manually or pay for a tool? (2026)

Yes, you can track AI visibility manually, and for your first baseline you should. Put 20 to 30 buyer questions in a spreadsheet and run each one three times a month in ChatGPT and one other engine. You'll see whether you're named, how you're described and which pages the answers cite. The catch is sample size. A sheet that size can show a big gap between you and a competitor, but it can't prove a five-point move. Once you track three or more engines, several markets or a trend for your board, a tool earns its fee. For most B2B teams, we recommend both.

Here at Searchmaxxed, we make law firms, clinics and premium builders the answer in Google and in the AI engines their buyers ask. We track what ChatGPT, Gemini, Perplexity and AI Overviews say about our clients, then do the work that changes it. So we'll be plain about our bias. Tracking only pays when someone acts.

Can you track AI visibility manually?

Yes. Manual tracking means asking AI engines your buyers' questions yourself and logging who gets named, how you're described and which pages get cited. It's slow, and it's the most honest read you'll get of what a buyer sees.

A clean manual check takes five habits:

1. Start clean. In ChatGPT, open a Temporary Chat and pick Unpersonalized before you type. OpenAI's help page says that mode skips your memory, custom instructions and plugins.
2. Ask like a buyer. Use unbranded questions ("best email security for a 500-person company") plus a few head-to-head comparisons.
3. New chat, every run. Earlier turns steer the next answer, so a repeat question in the same chat tells you little.
4. Log the context. Record the date, engine, country and run number beside every answer.
5. Save text, not screenshots. Paste the full answer and every cited URL into your sheet so you can search it later.

Reading is where the manual method wins. Say ChatGPT tells buyers your platform lacks SOC 2 when you passed the audit months ago, and a mention count scores it as a win. A person reading it spots a deal-killer and goes looking for the out-of-date page the engine trusted.

For the full setup, from picking questions to joining the data with Search Console, see our step-by-step guide to tracking AI visibility.

What can't a manual prompt sheet tell you?

A manual sheet can't give you a reliable rate, a small change or a clean trend. AI answers move between runs and from week to week, and one person can only log a few hundred answers a month.

It moves more than you'd think. Conductor ran each prompt 50 times in four AI engines; on purchase questions, only four in ten brands from any two runs appeared in both. SISTRIX tracked 82,619 prompts for 17 weeks and found ChatGPT replaced 74% of its cited sources each week, against 56% for AI Mode. AI Overviews barely moved, at 5%.

So one screenshot proves nothing.

Each engine also leans on its own mix of sources and its own index of the web. A sheet that only checks ChatGPT can't tell you what Perplexity or Gemini says about you, and those answers can name a different shortlist entirely.

A sheet has blind spots, too. There's no alert when a competitor appears on a Tuesday, and no history unless someone keeps the sheet going every single month.

How many answers do you need before the numbers mean something?

About 90 answers per engine puts your mention rate within roughly 10 points either way. Spotting a five-point move takes around 400. That's standard sampling maths, and it's the clearest reason AI visibility tools exist: they can log thousands of answers a month while nobody types.

We worked these out at 95% confidence for a rate near 50%, the widest case, using 1.96 × √(0.25 ÷ answers). They assume every answer is independent. Repeat runs of one question aren't, so treat each margin as the best you can hope for.

A worked example. Say you sell endpoint security and run 30 buyer questions three times each in ChatGPT, which gives you 90 answers. You're named in 18 of them (20%) and your main competitor in 54 (60%). That gap is real.

Next month you're named in 24 answers, or 27%, and it feels like progress. It sits inside the noise.

Now compare a tool. Otterly AI's Lite plan tracks 15 prompts daily across ChatGPT, Google AI Overviews, Perplexity and Microsoft Copilot for $29 monthly. If daily means one run per prompt per engine, that's about 1,800 answers a month. That's 20 times the worked example above. Confirm the run count with the vendor before you compare plans.

Is an AI visibility tool worth it?

An AI visibility tool is worth it once your tracking outgrows one person. That usually means more than about 200 answers a month, three or more engines, several markets, or a trend leadership wants every month. Below that, a manual sheet and the free Google and Bing reports do the job.

It's worth paying for a tool when any of these is true:

• Buyers shortlist vendors in AI chatbots. In G2's March 2026 survey of 1,076 B2B software buyers, 69% chose a different vendor than planned because of chatbot guidance.
• You need rates you can defend. Per engine, per market and per product line, with stored answers behind every number.
• Speed matters. A tool flags a new competitor within days, where a monthly sheet finds it weeks later.
• Several people need the same view. Shared dashboards beat a spreadsheet passed around by email.

Skip it, or wait, when:

• Nobody will act on it. A dashboard without someone fixing pages and earning mentions is a monthly bill for a chart.
• Your question set is small. Twenty questions in two engines fit comfortably in a sheet.
• You expect it to fix things. Tools report what the engines say. Changing it takes SEO, content and outside sources.
• Your buyers don't find vendors through AI. If deals come from referrals and outbound, an engine's opinion of you matters less.

One more limit applies to both. A tool tracks only the questions you give it, so a weak question set buys you precise, daily numbers about the wrong things.

What does an AI visibility tool cost compared with manual tracking?

Manual tracking costs nothing but time. Published entry plans for AI visibility tools ran from $29 to $99 monthly in November 2026, and enterprise plans are quote-only.

Prompt counts drive most of the price. Otterly's Standard plan lists 100 prompts for $189, Peec AI's Pro plan lists 150 for $245, and Semrush adds 50 prompts for $60 monthly. Profound quotes Enterprise on request.

The real cost of manual tracking is the hours. Time your first ten answers, from typing the question to logging the last cited URL, then multiply by the answers you plan each month. That number usually settles the question.

We compare 12 trackers side by side, with what each one can and can't measure, in our guide to AI visibility tools.

What should go in an AI visibility tracking spreadsheet?

An AI visibility tracking spreadsheet needs one row per answer, a summary tab and a margin-of-error column. That last column stops anyone celebrating noise.

Set up a Log tab with one row per answer and these 11 columns:

Then build a Summary tab with one row per month and engine, with the month in column A and the engine in column B. Add these formulas from row 2:

Swap "Rival" for a competitor's name. Call it a real change only when this month's rate sits outside last month's rate, plus or minus its own margin.

Read the cited-URL column first. It tells you where the work is, and in our category it pointed straight at a short list of pages:

Across 900 ChatGPT answers about Australian agencies, a single third-party "best agencies" site was cited in 168 of them. The next three were a second listicle, Semrush's agency directory and Clutch. If you want to be recommended, the job is getting onto the handful of pages the model already trusts.

What does a hybrid setup look like?

A hybrid setup lets a tool count and a person read. The tool runs your question set daily. Once a month, someone reads the full answers to your most commercial questions and checks the free Google, Bing and GA4 data.

The monthly routine we'd run:

1. Let the tool count. Track 50 to 150 unbranded buyer questions daily in the engines your buyers use, grouped by buying stage.
2. Read ten answers by hand. Use a clean session, note wrong facts and find the pages behind them.
3. Open Search Console. Since 31 August 2026, its Generative AI performance report counts every site's AI Overview and AI Mode impressions by page.
4. Open Bing Webmaster Tools. Its AI Performance report shows how often Microsoft Copilot and Bing's AI summaries cite your URLs, and which ones.
5. Check GA4 for visits. Its AI Assistant channel has grouped chatbot visits since 13 May 2026.
6. Ask your CRM. A "how did you hear about us" field catches buyers who read an AI answer and never clicked. Our guide to AI search attribution shows how to turn that into a pipeline number.
7. Refresh the questions each quarter. Add new ones from sales calls and Search Console, and keep most of the set fixed so your trend holds.

Free data needs checking too. Here's one client's GA4:

A client's GA4 showed AI-assistant visits jump from 354 to 6,050 in a month. Almost none of it was real. About 5,700 came in three one-hour bursts: same 1920x1080 desktop Chrome, all new users, no referrer, about 20 seconds each, landing on two pages. Real AI traffic was 385 sessions against 354 the month before, roughly flat.

Before you report a jump, look for one-hour spikes, a single screen size and missing referrers. Prompt tracking can't see that; someone has to read the analytics.

Is Searchmaxxed right for you?

We're a strong fit if you're a law firm, clinic or premium builder that's missing from AI answers or losing ground in Google. You want the answers to change, and you want the work done for you.

Here's what you get. Our managed programs include AI answer tracking across ChatGPT, Gemini, Perplexity and AI Overviews. The people doing your work read every report. We go after the searches and AI answers that bring buyers, then put your name on the pages those answers trust. We track which searches and AI answers turn into revenue, through your CRM as well as GA4. You talk to the team building your results, with no account manager in between. And you own your site, content and data from day one. Our AI search optimisation service page sets out the full scope, from technical SEO and content to digital PR.

One client came to us with a website that wasn't working. Three months later he was on page one for one of the most competitive searches in his industry. Now he has more high-intent leads than he can handle, and we're building the systems to handle them.

The bigger picture:

Searchmaxxed's work has generated more than $15 million in revenue through search for its clients.

"I can't believe this worked." That's what we hear. Clients tell us they finally found a growth partner in search that keeps its word and gets things done.

We're the wrong call in three cases. If you only want to watch your mentions, an AI visibility tool will cost you less. If you need leads this month just to survive, search takes longer than that, and if you want the cheapest retainer, look elsewhere.

What should you do this week?

Start small. Write 20 buyer questions and run each three times in a ChatGPT Temporary Chat set to Unpersonalized, logging who gets named. Then open Search Console's Generative AI report. If a competitor shows up where you don't, you know where to start. Our guide to showing up in AI search lays out the fixes in order, from crawler access to the sites AI answers trust.

Want us to do it with you? Get your free search analysis. We'll benchmark you against your top competitors across Google, ChatGPT and the other AI engines. We'll show you where they're named instead of you, and the few moves that will matter most.

Frequently asked questions