Modern SEO in 2026: From Keywords to AI Retrieval

For years, SEO was explained as a matching game: find the keywords people type, put them in the right places, earn links and wait. That picture was never complete, but it was close enough to work. Today it leaves out too much. Search engines interpret meaning as well as words, they answer questions directly on the results page, and a growing number of people ask AI tools for answers and get a written response with a handful of source links instead of ten blue links. Understanding how that changes the work is what people mean by “modern SEO”.

This guide is the overview for the rest of our modern SEO coverage. It explains what has changed, what has not, how information moves from your page to an AI-generated answer, which parts of the picture are documented by search engines and which are practitioner theory, and what you can do about it with free tools. It is deliberately cautious about claims. The field is full of confident statements that nobody can verify, and the honest version of modern SEO is more useful than the hyped one.

Key takeaways

• Modern SEO builds on the fundamentals: crawlable, indexable, useful pages. It adds attention to meaning, entities, clear passages, coverage of related questions and measurement of AI-driven visibility.
• Google states that no special optimisation is required for AI Overviews and AI Mode, that pages must be indexed and eligible to show with a snippet, and that existing SEO best practices still apply.
• Many modern SEO terms (query fan-out, retrieval, embeddings, entity salience) are real technical concepts, but claims that you can directly “optimise for” them are usually hypotheses.
• The most reliable improvements are the ones that help readers: clear structure, direct answers, specific evidence, consistent naming and complete coverage of what people need next.
• Measure honestly. Use Search Console, GA4 and a transparent observation log for AI answers, and avoid single-number scores whose method you cannot inspect.

How to read the evidence in this guide

We label each claim as one of four kinds: official documentation (stated by a search provider), established technical concept (well known in information retrieval or machine learning), practitioner convention (a method SEOs use that is not documented as how an engine works) or hypothesis. If a statement has no label, treat it as ours and test it.

What Is Modern SEO?

Modern SEO is search optimisation that accounts for how search systems now interpret queries and content, including the use of language models, structured knowledge and generated answers. It keeps the core goal of classic SEO, which is to make useful pages easy for search systems to find, understand and trust, and extends the work to cover how those pages are retrieved, summarised and cited.

It is not a replacement for technical, content and authority work. It is an additional lens. A page that is blocked from indexing cannot appear anywhere, however clever its AI-search strategy. A page that is indexed but vague, thin or untrustworthy will struggle in both classic and AI-driven results.

The acronyms, sorted

The vocabulary is crowded, and much of it overlaps. This table separates the terms and says how well supported each is.

Term	What it usually means	Evidence status
AI Overviews	Google’s AI-generated summaries on results pages, with links to sources	Official Google feature
AI Mode	Google’s conversational search experience for complex questions	Official Google feature
AIO	Used for both “AI Overviews” and “AI optimisation”, so check the context	Ambiguous term
AEO (answer engine optimisation)	Making content suitable to be used as a direct answer	Practitioner term; Google says optimizing for generative AI search is still SEO
GEO (generative engine optimisation)	Making content visible in generative answers	Practitioner term; also appears in research literature
LLMO (large language model optimisation)	Optimising for language-model-based tools	Practitioner term
Query fan-out	Generating related sub-searches to build an answer	Described in Google’s documentation for AI features
RAG	Retrieving documents to ground a generated answer	Established AI concept
Embeddings and vector search	Representing meaning as numbers and retrieving by similarity	Established technical concept
Entity salience	How central an entity is in a text	A Google Cloud API feature; no documented ranking use
Passage ranking	Assessing sections of a page for relevance	Documented Google ranking system

Our glossary entries on AI search, AI search optimisation and AI SEO give short definitions. This article connects them.

What has genuinely changed

• Queries are longer and more conversational. Google has described AI Mode as suited to queries needing exploration, reasoning or complex comparisons. (Official documentation.)
• Answers are assembled. Google says AI Overviews and AI Mode may use query fan-out, issuing multiple related searches across subtopics and data sources. (Official documentation.)
• Visibility is partly invisible. A page can inform an answer without receiving a click. Google introduced dedicated Search Console reports on 3 June 2026 and says they have been available to all websites worldwide since 31 August 2026 showing impressions in generative AI features, but they report impressions, not clicks. (Official documentation.)
• Other AI tools exist. Chat assistants and answer engines retrieve and cite pages using their own systems, each with its own documentation and crawlers.

What has not changed

• Pages must be crawlable and indexable to be eligible. Google says a page needs to be indexed and eligible to be shown with a snippet to be a supporting link.
• Helpful, reliable, people-first content remains the foundation.
• Technical health, internal linking and clear site structure still matter.
• Reputation and trust still matter, and they take time to build.

How Information Moves From Your Page to an AI Answer

It helps to picture a pipeline. Google does not publish every step, so this is a conceptual model built from official statements and general retrieval practice. Each stage shows where an SEO concept fits.

1. Discovery and crawling. The system finds and fetches your page. Robots rules, server health and internal links matter. AI tools may use their own crawlers, which you can allow or block through robots.txt, as each company documents. See our entry on AI crawlers.
2. Indexing. The page is processed and stored. Rendering, canonicals and duplicate handling matter.
3. Query understanding. A user asks something. The system works out intent and entities, and may generate related sub-queries (query fan-out).
4. Retrieval. Candidate pages or passages are fetched using keyword matching, semantic similarity or both.
5. Ranking and selection. Relevance, quality and other signals decide what is used. Passage-level assessment is one documented element.
6. Generation and attribution. A model writes an answer from the selected material and, in many products, shows sources.
7. Measurement. You see impressions, clicks, referral visits and mentions, partly and imperfectly.

Stage	Related concepts	What you can influence	What you cannot see
Crawling and indexing	Robots, rendering, sitemaps, canonicals	Eligibility and technical health	Internal crawl scheduling
Query understanding	Intent, entities, query fan-out	Coverage of the follow-up questions	The actual sub-queries
Retrieval	Keyword, vector and hybrid search; embeddings	Clear wording, precise terms and plain explanations	The models and weights used
Ranking and selection	Passage ranking, quality systems, links	Passage clarity, evidence, trust	Exact signals and weights
Generation and citation	RAG, attribution	Making claims specific and verifiable	Whether a source is shown
Measurement	Search Console, GA4, observation logs	Your tracking and logging discipline	True AI usage and reach

A Real-World Example

This example is illustrative. It shows how the same page can look different through the old and modern lenses. It uses an invented page and no real data.

A consultancy publishes a page titled “SEO Services”. The old lens asks: does it include the target keyword in the title, headings and body, and does it have links? The page does, and it ranks moderately.

The modern lens asks different questions:

• When someone asks “what should a small business expect from an SEO audit”, is there a section that answers it directly, with specifics? The page has marketing language and no answer.
• Are the follow-up questions covered: cost, timeline, what is included, how to choose? Only one is.
• Is the organisation named consistently and described accurately, here and on external profiles? The page uses three different brand variants.
• Is each section understandable alone? Several depend on “as above”.
• Can the page be crawled, indexed and shown with a snippet? Yes.
• Will anyone be able to tell whether it is working? There is no tracking of enquiries from organic search.

None of these fixes involves a trick. They are editorial, structural and measurement improvements that help readers and are consistent with how retrieval systems work. The modern lens simply makes the gaps visible.

Implementation Steps: A Modern SEO Review

1. Confirm eligibility. Check that important pages are indexed, return a normal status, are not blocked by robots rules and do not carry noindex or snippet restrictions that you did not intend. Use URL Inspection in Search Console for key pages.
2. Decide your AI crawler policy. Review your robots.txt and decide which AI crawlers you want to allow. Each provider documents its own crawler names and controls, and these change, so check the current documentation before editing. Blocking is a business decision with trade-offs.
3. Audit key pages for direct answers. For each priority page, list the three to five questions a buyer would ask and check whether each is answered clearly within the page.
4. Build a question map. Group the related questions by intent and decide where each is answered, using your existing URLs first.
5. Strengthen passages. Give sections descriptive headings, lead with the answer, name the subject, state conditions and include evidence.
6. Unify entity descriptions. Standardise how your organisation, services and people are named and described across your site and main profiles.
7. Check structured data. Make sure markup is valid, matches visible content and is appropriate for the page. Do not expect it to guarantee inclusion.
8. Set up measurement. Make sure GA4 conversions are tracked. Check Search Console for the generative AI reports if available. Create a prompt log for AI tools and decide how often you will run it.
9. Establish a baseline before changing content, so you can compare afterwards.
10. Fix, then review. Make changes in priority order, log them and review after six to eight weeks.

A note on llms.txt and similar files

Google’s guide says Google Search ignores llms.txt and similar files and that you do not need them to appear in Google Search, including its generative AI features. It is fine to maintain one for other services that read such files, and doing so will neither help nor harm your Google visibility. Do not present it to clients as a lever for Google. If another service you care about documents support for it, judge that on its own terms.

Free Tools

Tool	Use	Limits and checks
Google Search Console	Indexing, queries, pages and generative AI impressions where available	Not every query shown; AI report shows impressions, not clicks; confirm availability on your property
GA4	Organic and referral sessions and conversions	Needs correct set-up; some AI visits arrive without a referrer
Google Trends	Seasonality and relative interest	Not search volume
Google Search	Check intent, result types and AI features	Varies by location and personalisation
Google Sheets	Question maps, prompt logs and audits	Needs consistent entries
Screaming Frog (free version)	Crawl small sites for status codes, titles and links	Free version has a crawl limit; check the current limit
Bing Webmaster Tools	Another search index and its reports	Check which reports are available to you
Local Python tools (open source)	Small experiments with embeddings and retrieval	Requires technical skills; results do not describe commercial engines
An AI assistant such as Claude	Brainstorm questions, summarise exports and draft for review	Outputs are hypotheses; verify facts; free usage limits apply

What Google Says You Can Ignore, and What It Requires

In May 2026 Google published an official guide, Optimizing your website for generative AI features on Google Search, last updated in July 2026. It is the most authoritative source on this topic, and several of its statements settle questions that vendors and commentators had argued about for months.

• AEO and GEO are still SEO. Google says that, from its perspective, optimizing for generative AI search is optimizing for the search experience, and so is still SEO. If you are considering third-party AEO or GEO advice, it points you to its guidance on evaluating third-party SEO advice.
• You do not need llms.txt or similar files. Google says you do not need new machine-readable files, AI text files, special markup or Markdown to appear in Google Search, including its generative AI features, because Google Search does not use them. Keeping such files for other services is fine and will neither help nor harm your Google visibility.
• Chunking content is not required. Google says there is no requirement to break content into tiny pieces and no ideal page length, and that its systems can understand several topics on a page and show the relevant piece. Clear sections still help readers.
• You do not need to rewrite content for AI systems. Google says AI systems understand synonyms and general meanings, so you do not need to chase every long-tail variation.
• Inauthentic mentions are not worth seeking. Google says its core ranking systems focus on high-quality content while other systems block spam, and its generative AI features depend on both.
• Structured data is not required for generative AI search, and there is no special schema.org markup for it. It remains useful for rich results.

What it does require. To be eligible to appear in generative AI features, a page must be indexed and eligible to be shown in Google Search with a snippet, meeting the Search technical requirements. In addition, the site must be included in the Search generative AI control in Search Console. Inclusion is the default for all properties, and Google says it rolled the control out to all websites worldwide on 31 August 2026. The control covers AI Overviews, AI Mode and generative AI features in Discover, and Google says it is not used as a ranking or inclusion signal for other parts of Search. Limiting AI training is a separate matter, handled with Google-Extended.

A warning about mass-producing pages. Google says that creating separate content for every possible variation of how people search, including fan-out queries, primarily to manipulate rankings or generative AI responses violates its scaled content abuse spam policy, and that a high quantity of pages does not make a site more relevant. Plan coverage for readers, not for imagined sub-queries.

How to measure. Google recommends the Generative AI performance report in Search Console, and cautions against third-party tools that claim to use internal Google metrics.

Limitations and Misconceptions

• “There is a secret AI ranking factor.” No search provider has documented one. Be sceptical of anyone selling a trick.
• “SEO is dead.” Pages still need to be found, indexed and trusted, and classic results remain a major part of search. What has changed is the mix and the ways people reach answers.
• “Schema guarantees citations.” Structured data helps systems understand a page. It does not guarantee inclusion in any feature.
• “AI visibility can be measured with one score.” Results vary by run, platform and user. Any score needs a transparent method.
• “Write short chunks for AI.” Clear sections help, but fragmenting a page to chase retrieval can make it worse for readers.
• Behaviour differs across products. Google says AI Overviews and AI Mode may use different models and techniques, and other AI tools differ again.
• Things change quickly. Documentation, features and reports evolve. Date-check anything you rely on.

A Measurable Experiment

Hypothesis (unverified): Updating a set of important pages so that they answer buyer questions directly, name entities consistently and are easy to crawl will increase impressions and distinct queries in Search Console and the frequency of citations in a fixed prompt set, compared with similar pages left unchanged.

1. Select pages. Choose six to ten priority pages of similar type and split them into treatment and control groups.
2. Baseline. Export 28 days (ideally eight weeks) of Search Console data for each page, including generative AI impressions if available. Run a fixed set of 20 to 30 prompts across your chosen AI tools, three runs each, and log mentions and citations. Record GA4 organic and AI-referral sessions and conversions.
3. Intervention. On treatment pages only, apply the review steps above: direct answers, passage clarity, coverage of the key follow-up questions and consistent naming. Log each change and the date.
4. Observation window. Wait six to eight weeks.
5. Re-measure under identical conditions: the same date length, prompts, platforms and run counts.
6. Compare treatment and control groups, and each group with its baseline.

Primary metrics: Search Console impressions and distinct queries per page; citation count across the prompt runs. Secondary: clicks, generative AI impressions, AI referral sessions and conversions. Guardrails: organic conversions on treated pages should not fall. Confounders: seasonality, search and AI product updates, new links, competitor changes, run-to-run variation in AI answers and a small sample. Establish your normal run-to-run variation first, because a change smaller than that variation is not evidence. Interpretation: describe results as observations under stated conditions. A positive difference between treatment and control is encouraging. It does not prove that any particular technique caused it.

A Modern SEO Scorecard You Can Reuse

A scorecard turns the review into something repeatable. Score each row as pass, partial or fail for a priority page or for the site as a whole, and record the date. The criteria are practitioner checks that follow from official guidance and general retrieval practice. They are not a Google score.

Area	What to check	Pass looks like	Basis
Eligibility	Index status, robots rules, noindex, snippet limits	Priority pages indexed and eligible for snippets	Official guidance
Crawler policy	Robots rules for search and AI crawlers	A documented, deliberate decision	Provider documentation
Answer clarity	Direct answers to the key buyer questions	Each question answered in a labelled section	Practitioner convention
Coverage	Follow-up questions mapped to pages	Each important question has one clear home	Practitioner convention
Entity consistency	Names, descriptions and facts across site and profiles	One name and one description everywhere	Practitioner convention
Evidence	Sources, examples and first-hand detail	Claims supported or clearly flagged	Quality guidance
Structure	Headings, internal links, self-contained sections	Sections understandable alone	Practitioner convention
Measurement	Conversion tracking, Search Console, prompt log	Baseline recorded and reviewed on a schedule	Good practice

Use the scorecard to decide where to start. A fail on eligibility outranks everything else, because nothing downstream can work if a page cannot be indexed. A fail on measurement comes second, because without a baseline you cannot learn from the other changes.

How to Talk About Modern SEO With Clients and Stakeholders

Stakeholders often arrive with strong, confused beliefs, usually from headlines or vendor pitches. A calm, specific explanation works better than either hype or dismissal.

• Start with what is documented. Say what Google has stated about AI features, and show the source.
• Separate fact from theory. Explain that concepts like fan-out and retrieval are real, while claims about controlling them are unproven.
• Offer a test, not a promise. Propose a controlled experiment on a small set of pages with clear measures and a review date.
• Be clear about visibility limits. Explain that some AI visibility cannot be measured directly, and show what you will track instead.
• Keep the foundations in view. Reassure them that technical health and useful content remain the base, so no budget needs to be abandoned.

This framing keeps you credible. If a claim later turns out to be wrong, you have not staked your reputation on it, and if an approach works, you have the evidence to show why.

Common Mistakes

• Rebranding old tactics as AI optimisation. Stuffing synonyms or adding an FAQ to every page is not a strategy.
• Skipping the fundamentals. Indexing, speed, structure and trust come first.
• Creating pages for every possible question. This causes duplication and cannibalisation. Check your existing URLs first.
• Presenting hypotheses as facts. Label what is documented and what is not, to clients and in your own notes.
• Ignoring measurement. Without a baseline and a log, you cannot tell whether anything worked.
• Over-trusting tools. Treat tool outputs as prompts for investigation, not verdicts.

Frequently Asked Questions

Do I need a separate strategy for AI search?

Not a separate one. You need a strategy that covers classic search and AI-driven discovery with the same foundations: indexable, useful, trustworthy content, plus attention to coverage, clarity and measurement.

What is the difference between AEO, GEO and LLMO?

They are overlapping practitioner terms for making content visible in AI-generated answers. There is no standard definition separating them, so ask anyone using them to explain exactly what they mean.

Does Google require anything special for AI Overviews or AI Mode?

Google states there are no additional requirements or special optimisations. The page must be indexed and eligible to appear with a snippet, and normal SEO best practices apply.

Can I see my performance in AI Overviews?

Google introduced Search Console reports for generative AI features on 3 June 2026, showing impressions, pages, devices and dates. Check whether it is available on your property.

Should I block AI crawlers?

It depends on your goals and risks. Review each provider’s documentation, understand what each crawler is used for and make a deliberate decision. Blocking some crawlers can reduce your visibility in those products.

Will this guarantee citations?

No. No one can guarantee citations. The steps improve usefulness and clarity, and the experiment shows whether they appear to help your site.

How often should I review this?

Check official documentation and your measurements at least quarterly, because features and reports change quickly.

Sources and Further Reading

• Google Search Central: AI features and your website
• Google Search Central Blog: Introducing Search Generative AI performance reports in Search Console (3 June 2026)
• Google Search Central: Optimizing your website for generative AI features on Google Search
• OpenAI: crawler documentation. Anthropic, Perplexity and other providers publish their own crawler documentation, so check each provider’s current page

Related reading

• How crawling, indexing and ranking work
• What is query fan-out?
• How to measure AI search visibility

Next Step

If you want a modern SEO review of your site covering eligibility, answer coverage, entity consistency and AI visibility measurement, request a free SEO audit. You can also see how we work on our services page.