SEO for AI search is the same job as SEO, with three extra checks. Let the right bots in. Write pages that are worth quoting. Measure what you can. That is the short answer.
Google says the same in its own guide. It says optimizing for AI features on Google Search is still SEO. This page walks through each part in plain words. I read the official pages on 15 Nov 2026 and I link every one of them.
We sent requests to our own site using eight bot names and show what came back. We have not tested whether any change gets a page cited. Nobody outside the AI companies can promise that, and neither can we.
On this page19 sections
What AI search is, in plain words
AI search means a tool answers your question in your own words, then shows links to the pages it used. Google has AI Overviews and AI Mode. ChatGPT, Claude, Perplexity, Copilot and Gemini do similar work in their own apps.
Google explains how its two features work. It uses two ideas, and both are worth knowing.
- Retrieval-augmented generation (RAG). Google also calls it grounding. The model first fetches pages from Google’s search index. Then it writes the answer from those pages and shows links.
- Query fan-out. The model asks several related searches at once. Google’s example is a question about fixing a weedy lawn. The extra searches might ask about the best weed killers or how to stop weeds coming back.

The big lesson is simple. For Google, the answer starts from the same index and ranking systems as normal search. A page that is not indexed cannot be a source.
Other AI tools have their own bots and their own rules. They do not publish how they pick sources. We cover what they do say below.
The alphabet soup: SEO, AEO, GEO, LLMO and AIO
You will see many short names for this work. Most mean nearly the same thing. Here is what each one means and where we could check it.
| Name | What it means | What we could check |
|---|---|---|
| AI SEO | SEO that also aims at AI answers | Search Engine Land uses it as the umbrella term |
| GEO | Generative engine optimization | Named in a 2023 research paper by Aggarwal and others, accepted to KDD 2024 |
| AEO | Answer engine optimization | Google’s guide names it. Sources date its start differently, so we give no year |
| LLMO | Large language model optimization | Listed by Search Engine Land with GEO, AEO and AIO |
| AIO | AI optimization, or Google’s AI Overviews | Used both ways, so check the meaning each time |
| Agentic SEO, SEO for agents | Making a site work for AI agents that act for a person | Google’s guide has a section on agentic experiences |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.

Search Engine Land says GEO, AEO, LLMO and AIO mean essentially the same thing. Google goes further. Its guide has a short note on AEO and GEO, shown here.

You may also see other labels. We found one writer who uses "IEO" for influence engine optimization. We found no settled meaning for it, so we do not use it. If a label has no clear definition, ask the person using it to explain.
The research paper is real, and it is the oldest source we found for the GEO name. Its abstract says the authors tested ways to improve visibility in AI answers and saw gains of up to 40 percent in their tests. It also says results differed by topic. Read it as one study, not a rule. The paper is on arXiv.
What Google says in its own guide
Google published a guide to generative AI features in Search. The page says it was last updated on 10 July 2026. Here is what it says, in short.
- SEO still applies. The AI features sit on Google’s core ranking and quality systems.
- The only technical rule: a page must be indexed and able to show with a snippet. The site must also be included in Search generative AI features in Search Console.
- Content matters most. Google says unique, helpful content will likely influence your presence in AI search more than any other tip in the guide.
- Non-commodity content. Google gives an example. A list of tips for first-time homebuyers is common knowledge. A first-hand story about waiving an inspection, and what it showed, adds something new.
- No page spam. Making a separate page for every fan-out query, mainly to game results, breaks Google’s scaled content abuse policy.
Google also lists things you can ignore for Google Search. This list surprises many people.
| The idea | What Google says |
|---|---|
| llms.txt files and special markup | Google Search does not use them. Making one will not help or harm you there |
| Chunking content into tiny pieces | No requirement. Google can find the right part of a page |
| Rewriting pages just for AI | Not needed. AI systems understand synonyms and meaning |
| Seeking fake mentions | Not as helpful as it seems. Ranking systems focus on quality and block spam |
| Over-focusing on structured data | Not required for AI search, and no special markup is needed |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.

A fair question: does this apply to ChatGPT, Claude and Perplexity? Google’s rules are for Google. The other tools publish bot pages but no ranking recipe. So be careful with anyone who sells a sure-fire method for them.
How AI tools choose what to cite
Honest answer: from the outside, nobody knows exactly. Google explains its process in general terms. OpenAI, Anthropic and Perplexity publish how their bots behave, not how they rank pages.
What we can say is what the guides and the research suggest. Treat each point as an idea to test, not a promise.
- Be easy to fetch. If a bot cannot read your page, it cannot use it.
- Give a clear answer early. Search Engine Land suggests direct answers near the top of each section.
- Add something original. Google asks for first-hand experience and a unique view.
- Show your sources and dates. Trust signals help readers, and AI answers often pull from pages that look careful.
- Be talked about, for real. Search Engine Land notes that AI tools read forums and video as well as sites. Real mentions are normal. Fake ones do not help, per Google.
- Keep it fresh. Update old posts when facts change.
Several big SEO sites have written long guides on this. Ahrefs, Semrush and Similarweb each show their own data. Many of their numbers come from their own tools, so read them with that in mind.
Let the right bots in: robots.txt
This is the most practical step. Each company runs more than one bot, and each bot has a different job. Mixing them up is the most common mistake.

| Bot | From | Its job | If you block it |
|---|---|---|---|
| Googlebot | Search, including AI Overviews and AI Mode | Google cannot crawl your pages for Search | |
| Google-Extended | Training future Gemini models, and grounding in Gemini Apps | No effect on Search inclusion or ranking | |
| OAI-SearchBot | OpenAI | ChatGPT search results | You will not show in ChatGPT search answers, only as a link at most |
| GPTBot | OpenAI | Training generative AI models | Your content is not used for training |
| ChatGPT-User | OpenAI | Fetches a page when a person asks | OpenAI says robots.txt rules may not apply |
| ClaudeBot | Anthropic | Training data | Future material is left out of training |
| Claude-SearchBot | Anthropic | Search quality for Claude | Anthropic says visibility may drop |
| Claude-User | Anthropic | Fetches a page when a person asks | Anthropic says visibility may drop for user-directed search |
| PerplexityBot | Perplexity | Surfaces sites in Perplexity search, not training | You may not show in its search results |
| Perplexity-User | Perplexity | Fetches a page when a person asks | Perplexity says it generally ignores robots.txt |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.
OpenAI says each setting is independent. You can allow OAI-SearchBot to appear in ChatGPT search, and disallow GPTBot so your content is not used for training. OpenAI also says a robots.txt change can take about 24 hours to take effect for search.




An example robots.txt
This is only an example, not a recommendation. It lets search bots in and asks training bots to stay out. Think about your own goals first.
# Let AI search bots in
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
# Ask training bots to stay out (optional)
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
Two warnings. First, blocking a search bot can take you out of that tool’s results. Second, if you use a firewall or CDN, it may block bots before robots.txt is read. Perplexity’s page explains how to allow its bots in a firewall, and OpenAI asks you to allow its published IP ranges.
What we found on our own site
Here is our real robots.txt, read on 15 Nov 2026. It has no rules for AI bots. It only blocks admin, cart and tracking paths.
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /go/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Disallow: /*?add-to-cart=
Disallow: /*?*add-to-cart=
Disallow: /wp-content/uploads/wc-logs/
Disallow: /wp-content/uploads/woocommerce_transient_files/
Disallow: /wp-content/uploads/woocommerce_uploads/
Sitemap: https://www.digitalpritam.in/sitemap_index.xml
Then we sent five requests for one page with each bot name, from a home computer in India. This tests how the server treats the name. It does not prove what the real bot sees, because real bots come from their own published IP addresses.
| Bot name sent | Result, five tries |
|---|---|
| GPTBot | HTTP 429 every time, with an empty page |
| OAI-SearchBot | HTTP 200 every time |
| ChatGPT-User | HTTP 200 every time |
| ClaudeBot | HTTP 200 every time |
| Claude-SearchBot | HTTP 200 every time |
| PerplexityBot | HTTP 200 every time |
| Googlebot | HTTP 200 every time |
| A normal browser | HTTP 200 every time |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.
The odd one is GPTBot. The server answered 429, which means “too many requests”, on all five tries. The other names were fine. We cannot tell if the real GPTBot gets the same answer. If you see this on your own site, ask your host whether it limits that bot name.
You can run the same test yourself. Change the name and the address. Do not use it against sites you do not own.
curl -s -o /dev/null -w "%{http_code}\n" \
-A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
https://example.com/
llms.txt: what is true and what is not
An llms.txt file is a plain text file at the root of a site. It gives AI tools a short summary and a list of key pages. Jeremy Howard proposed it on 3 September 2024. The llmstxt.org page calls it a proposal, not a ratified standard.

- Google: its guide says Google Search does not use llms.txt. It says making one will neither help nor harm your visibility there.
- OpenAI, Anthropic, Perplexity: we found no official page saying their search bots read it. That is a gap in what we found, not proof they ignore it.
- Other uses: some coding tools and documentation platforms use it. That is a different job from ranking.
We publish one on our own site, shown below. We did not make it to rank in Google. Treat ours as a small extra, and keep it accurate. A file with old prices is worse than no file.

Structured data, entities and mentions
An entity is a person, brand or thing that tools can recognise. AI tools read what is said about your brand across the web. So say the same clear facts in the places you control.

- An About page that says who you are and what you do.
- Author pages with a real name, a short bio and real links.
- The same facts everywhere: name, role and links on your site and your social profiles.
- Real mentions from reviews, press and communities. Do not buy or fake them.
Google says structured data is not required for AI search. It is still part of normal SEO, and Google asks that it matches what people can see on the page. Here is a trimmed piece of our real author markup. It lists the author’s profiles so tools can link them together.
{
"@type": "Person",
"@id": "https://www.digitalpritam.in/about/#pritam",
"name": "Pritam Sonone",
"url": "https://www.digitalpritam.in/about/",
"jobTitle": "Creator and growth strategist, founder of Digital Pritam",
"sameAs": [
"https://profiles.wordpress.org/digitalpritam/",
"https://www.linkedin.com/in/pritamsonone/",
"https://www.youtube.com/@DigitalPritam",
"https://x.com/digitalpritam_",
"https://www.instagram.com/digitalpritam_/"
]
}
This is our live markup, trimmed to five links. Adding it did not guarantee anything. It is housekeeping.
Write pages AI tools can quote
Google’s advice here is simple. Make content people find useful and not available everywhere else. These habits fit both people and AI.
- Answer first. Put the short answer in the first lines of a section, then explain.
- Use clear headings. Make each heading say what the section answers.
- Write short sentences. Explain a hard word the first time you use it.
- Add what only you know. A test you ran, a screenshot you took, a mistake you made.
- Show who wrote it and when. Add an author line, a date and sources.
- Use tables for facts. A small table is easy for people and tools to read.
- Keep pages up to date. Fix old prices and old advice.
Here is a small example of the difference. Both lines are examples we wrote, not quotes.
| Commodity line | Non-commodity line |
|---|---|
| Page speed is important for SEO | We cut a page from three big images to one, and our own PageSpeed run showed the change. Here is the report |
| Add FAQs to your page | These five questions came from real emails we got in November. Here is how we answered each one |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.
Make your site easy for AI agents
An AI agent is a tool that does a task for a person, like comparing products or filling a form. Google’s guide has a short section on this. It says browser agents may look at screenshots, the page structure and the accessibility tree.
Google links to a guide on web.dev with practical advice. In plain words, it says this.
- Use real buttons and links, not styled boxes that only look like them.
- Label every form field and connect the label to the field.
- Do not cover buttons with see-through layers.
- Make controls big enough to find and press.
- Keep key buttons in the same place on every page.
- Think about the experimental WebMCP standard, which web.dev says may change.
This helps people who use screen readers too. Google also mentions new protocols such as the Universal Commerce Protocol, so watch that space if you sell online.
A 15 minute WordPress check
Most readers here run WordPress. Do these steps in order. We have not recorded them on a fresh site, so check each screen against your own.
- Open yoursite.com/robots.txt in a browser. Read it. Make sure it does not block the search bots you want.
- In Settings, Reading, make sure the box that discourages search engines is off.
- Look at your SEO plugin. Most let you edit robots.txt. Add rules only if you need them.
- Check your firewall or CDN. If you use one, look for bot rules that may block AI search bots.
- Open Search Console and make sure your sitemap is sent and your key pages are indexed.
- Check that your key pages load fast on a phone and show their text without a click.
- Add an author line and an About page link on your best posts.
- Read our SEO tips and WordPress SEO plugin comparison for the basics.
How to measure AI visibility
Measuring is the weak spot of this whole field. Each tool shows a slice. Use several and be careful with the numbers.
Search Console: Generative AI performance report
Google built a report for this. It shows how often links to your site were shown in AI Overviews and AI Mode.
- It counts impressions. It does not show clicks or position.
- It covers AI Overviews and AI Mode, with a separate view for Discover.
- You can group by page, country, date and device. Dates use Pacific Time.
- You can filter by text search or by multimodal search such as Lens.
- Google says it rolled these insights out to all sites on 31 August 2026.

Google also says AI feature traffic is part of the overall search traffic in the Web search type of the Performance report. So you cannot split classic and AI clicks from the totals.
GA4 and Bing
- GA4: several publishers report that Google Analytics added an AI Assistant channel on 13 May 2026. They say it groups visits from ChatGPT, Gemini and Claude by the referrer, and that visits with no referrer stay in Direct. We did not open Google’s own help page for this, so check it in your own GA4.
- Bing Webmaster Tools: it has an AI Performance report. Bing’s own blog from 16 June 2026 adds four preview features: Intents, Topics, Citation Share and Compare. Coverage says the report shows citations, not clicks.
Check by asking
You can also ask the tools directly, and note what they say. Use a fresh chat, and ask the same question on a few tools. Write down the date.
1. What is the best way to [your topic] for a beginner in India?
2. Which sites explain [your topic] clearly? Give links.
3. Compare [your product] with [a rival] in a table.
4. What are common mistakes with [your topic]?
5. Who is [your name or brand]? What are they known for?
Answers change from one run to the next. Treat this as a rough check, not a score.
For agencies: how to sell AI search work honestly
Many clients now ask about GEO and AEO. Google says it has no hacks that guarantee a spot, and it points to its own advice on judging third-party SEO services. So sell work you can show.
| Service | What you deliver | How you show proof |
|---|---|---|
| Access audit | Which bots can reach which pages | A list of bot names with the result for each |
| Indexing check | Which key pages are indexed and eligible for a snippet | Search Console status per page |
| Content upgrade | Original, first-hand pages with clear author facts | Before and after pages, dated |
| Entity clean-up | Same facts on About, author and profile pages | A one page fact sheet and the links |
| Measurement setup | Reports in Search Console, GA4 and Bing | A dashboard with impressions and AI referrals |
| Prompt checks | A set of questions asked on a schedule | A dated table of answers and sources |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.
- Do say: we fix access, content and measurement, and we report what changed.
- Do not say: we will get you into ChatGPT or Google’s AI answers.
- Do not sell: fake mentions, mass pages for fan-out queries or llms.txt as a ranking trick.
Use a plain monthly report. Here is a simple layout.
Month: November 2026
1. What changed: pages updated, bots allowed, facts fixed
2. Search Console: AI Overviews and AI Mode impressions, top 10 pages
3. GA4: visits from AI assistants, if the channel shows
4. Bing: citations and top pages, if available
5. Prompt checks: date, tool, question, did we appear (yes or no), source
6. Next month: three actions
A 30 day plan

| Week | Do this |
|---|---|
| 1. Open the doors | Read your robots.txt. Test bot access. Check key pages are indexed. Allow the search bots you want |
| 2. Fix the page | Rewrite the top five pages with an answer first, clear headings and short sentences |
| 3. Add proof | Add original examples, dates, sources, an author line and an About link |
| 4. Measure | Open the Generative AI report, set up GA4 and Bing, run your prompt checks, write the first report |
Bot names and jobs from OpenAI, Anthropic, Perplexity and Google crawler pages, read 15 Nov 2026.

The funnel shows where a page can drop out. Most fixes are near the top: can bots fetch it, and is it indexed?
Mistakes to avoid
- Blocking the wrong bot. Blocking a search bot can take you out of that tool’s results.
- Mixing up training and search. They are different bots with different jobs.
- Chasing llms.txt. Google says it ignores the file.
- Buying mentions. Google says fake mentions are not as helpful as they seem.
- Writing a page for every fan-out query. Google calls that a spam risk.
- Trusting one number. Each report shows a slice. Compare several.
- Promising results. No one can guarantee a citation.
Tools that can help
You can do everything above for free. A paid tool helps when you want daily tracking in one place. We compare several in our guide to AI SEO tools and in SE Ranking vs Semrush. If you want to try a rank tracker, SE Ranking has a 14-day free trial with no card, per its pricing page read on 15 Nov 2026.
How we chose
Frequently asked questions
What is SEO for AI search?
It is normal SEO plus a few checks for AI answers: let the right bots in, publish pages worth quoting, show who wrote them, and measure what you can. Google says optimizing for its AI features is still SEO.
What is the difference between GEO, AEO and SEO?
GEO means generative engine optimization and AEO means answer engine optimization. Both describe work to be shown in AI answers. Google says that from its side this is still SEO, and Search Engine Land says the terms mean much the same thing.
Do I need an llms.txt file?
Not for Google. Google says Google Search does not use llms.txt files, and that making one will neither help nor harm your visibility there. We did not find an official statement from OpenAI, Anthropic or Perplexity that their search bots read it.
Should I block GPTBot or ClaudeBot?
That is your choice. GPTBot and ClaudeBot are for training. Blocking them says you do not want your pages used for training. It is a different bot, such as OAI-SearchBot or Claude-SearchBot, that decides if you show in their search.
How do I get cited by ChatGPT?
OpenAI says to allow OAI-SearchBot in robots.txt and to allow its published IP ranges. Beyond that, nobody outside OpenAI can promise a citation. Make pages that are useful, clear and original, and check what ChatGPT says about your topic.
Does Google AI Overviews need special schema or markup?
No. Google says structured data is not required for generative AI search and that no special schema.org markup is needed. It still suggests keeping structured data that matches the visible text, as part of normal SEO.
How do I track AI traffic?
In Search Console, use the Generative AI performance report. It shows impressions from AI Overviews and AI Mode, not clicks. In GA4 and Bing Webmaster Tools there are newer reports, described below.
Is AI search killing SEO?
Search Engine Land says GEO, AEO and the other new names are not replacing SEO. Google says its AI features use its core ranking and quality systems. The work is changing, so measure more than clicks.
What can an agency honestly promise for AI search?
Work, not results. Google says there are no AEO or GEO hacks that guarantee a spot. An honest agency fixes access and content, sets up measurement, and reports what changed.
Where can I read the official rules?
Read Google’s guide to generative AI features, and the crawler pages from OpenAI, Anthropic and Perplexity. We link all of them at the end of this page.
About this page

- WordPress developer and designer, builds sites for clients
- WordCamp Asia Communication Co-Lead
- 14K+ subscribers on the DigitalPritam YouTube channel
How we checked
Updated 15 Nov 2026How I got these facts
I read each official page on 15 Nov 2026. Where a fact came only from news or blog coverage, the page says so. The bot name test used five requests per name from one computer, so it shows the server’s answer to a name, not what a real bot receives.
Sources
- Google Search Central: Optimizing your website for generative AI features on Google Search (last updated 10 July 2026), read 15 Nov 2026
- Google Search Central: AI features and your website (last updated 10 Dec 2025), read 15 Nov 2026
- Google Search Central: List of Google’s common crawlers (last updated 30 Oct 2026), read 15 Nov 2026
- Search Console Help: Generative AI performance report, read 15 Nov 2026
- OpenAI: Overview of OpenAI crawlers, read 15 Nov 2026
- Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler, read 15 Nov 2026
- Perplexity: Perplexity crawlers, read 15 Nov 2026
- llmstxt.org: The /llms.txt file (version 2, modified 10 Aug 2026), read 15 Nov 2026
- arXiv 2311.09735: GEO: Generative Engine Optimization (Aggarwal and others, KDD 2024), read 15 Nov 2026
- web.dev: AI agent site UX, read 15 Nov 2026
- Bing blog: New AI Visibility Insights in Bing Webmaster Tools, 16 June 2026, read 15 Nov 2026
- Search Engine Land: SEO vs. GEO, AEO, LLMO (updated 21 Sept 2026), read 15 Nov 2026
- Ahrefs, Semrush and Similarweb guides to generative engine optimization, read for structure on 15 Nov 2026
- SE Ranking pricing (seranking.com/pricing), read 15 Nov 2026
What to do next
Open your robots.txt today. Check that the search bots you want can reach your best pages. Then pick one page and make it the clearest answer on the web.
Try the free trial, and check the price on the day you buy. Plans change.
FREE PDFGet the Hostinger setup checklist, plus every sale alert
The exact steps we use on our lab sites, and an email when each big hosting sale opens.
Sent instantly. Deal alerts until 1 Dec. Unsubscribe in one click.


19 SEO tips
How to do keyword research
What is SEO?
AI SEO tools