How AI Search Works: Retrieval, Generation and RAG Explained

When you ask a question to an AI search system like AI Overviews or ChatGPT, the answer feels instant. You type something, you get an answer. But in that fraction of a second, something happens under the hood that is fundamentally different from classic search engines. And if you want your content to be picked up by these systems, you need to understand how that machine works.

In this article I explain the AI search architecture in plain language: from the hallucination problem that sets everything in motion, through query fan-out and the retrieval pipeline, to what the model really sees of your page. This is the technical foundation under the ultimate GEO guide, where GEO stands for generative engine optimization.

Apply it right away: let our GEO check show you which AI signals your page is missing.

Why AI search exists: the hallucination problem

Large language models (LLMs) are trained on enormous amounts of text. They learn patterns and connections, which lets them generate fluent-sounding answers. But that is exactly where the limitation sits: a model only “knows” what was in its training data, and it has no reliable way of knowing whether it truly knows something or is simply making up something plausible.

That is the hallucination problem. Ask a model about a recent event after its training cut-off, and it may invent a credible but entirely fictional answer. The fluent style says nothing about accuracy.

For creative writing that is manageable. For search, where people expect accurate and current information, it is unacceptable. The solution is called grounding: supplementing the model’s parametric knowledge (what sits in the weights) with retrieved knowledge (what is pulled from external sources in real time). A grounded answer effectively says: “I just found this, from these sources, and I am summarising it for you.”

Grounding turns a language model into something resembling a research assistant: it can consult current information, cite sources and admit when it does not know something. This is the architecture behind AI Overviews and every serious AI search implementation. If you want to dig deeper into this mechanism, read my article on grounding. The broader technique where retrieved knowledge and generation come together is called RAG; what RAG (retrieval-augmented generation) means for marketers, we explain separately.

Query fan-out: how AI googles itself

This is where the architecture gets genuinely interesting.

In classic search, your exact query is matched against an index. One query, one search action, one result set. AI search works differently: faced with a complex question, the model does not search once, but many times.

Take this question: “What are the best strategies to improve visibility in AI search for e-commerce brands?” An AI system recognises that this question has several dimensions and breaks it into sub-questions, for example:

• What is AI search?
• How does AI search differ from classic search?
• What challenges do e-commerce brands face specifically?
• Which optimisation strategies exist for AI visibility?
• Which examples or cases demonstrate success?

The model runs those sub-questions (sometimes sequentially, sometimes in parallel) against a search infrastructure. Each sub-question returns its own results, and then the model synthesises everything into a single answer. This is called query fan-out: the question “fans out” into multiple search actions.

The impact is significant. In the past you optimised for specific keyword queries. Now your content can be retrieved for questions you never anticipated. Your page about “conversion optimisation in webshops” can surface in an answer about “AI visibility strategies”, because the model made a relevant connection. The optimisation surface therefore becomes far broader than it used to be.

The retrieval pipeline: from thousands of URLs to one answer

What happens after the model has generated its fan-out questions? The information runs through a pipeline with four stages.

That pipeline works like a funnel: each stage prunes the candidates further, from thousands of URLs that could be relevant down to the handful of passages that ultimately form the answer.

Stage 1: fast retrieval

The sub-questions hit a search infrastructure. At Google that is a light, cached version of the index, optimised for speed. This is not the full search experience with all ranking signals, but a fast layer that returns candidate documents quickly. Think of it as a first sweep: gather everything that could be relevant, quality comes later. This stage may produce thousands of candidate URLs, far more than can ever be used.

Stage 2: filtering

A classifier assesses those candidates and filters them down to a manageable set. That happens on the basis of relevance, authority, freshness and other signals. But it is not classic ranking, more like triage: deciding which candidates deserve a closer look.

This stage is often underestimated. Pages that score well in classic search sometimes drop out here because they do not meet the quality threshold for grounding. Conversely, pages that rank low in classic search do make it through the filter, because their content is well structured for extraction. The filter criteria are therefore not identical to ranking criteria, and that is precisely why classic rank tracking only tells half the truth about your AI visibility.

Stage 3: content extraction

For the pages that survive the filter, the system fetches the content and pulls out the relevant passages. Here your carefully designed page is reduced to text blocks (chunks).

The model does not see your page the way a visitor does. No hero image, no brand colours, no navigation. It gets something closer to raw text, sometimes with basic HTML structure such as headers and bold text, but in essence just words. If your meaning depends on visual context (an infographic without alt text, meaning conveyed through layout rather than language), the model does not pick that meaning up.

Stage 4: selection and synthesis

Finally, the model assesses the extracted passages against the original question and chooses which ones it includes in the answer. That is the selection moment: does your content become part of the answer, or does it go unused?

Selection is not binary. The model can quote your content extensively as the main source, or take a single sentence from it mixed with five other sources, or retrieve your passages and still decide that alternatives are better. That is how selection rate becomes measurable: how often is your content, once retrieved, actually selected as well?

What the model really sees of your page

This point is often misunderstood, and that misunderstanding leads to the wrong optimisation. Many people think AI takes some sort of screenshot of your page. That is not right. The model sees text, and specifically text that has been pulled through the pipeline and stripped of visual formatting.

In some implementations basic HTML structure survives: the model sees that text sat in an <h2> or was bold, and recognises lists. But fonts, colours, spacing and images do not survive. Schema markup and structured data sit in a grey zone: some systems use them, others ignore them.

The practical lesson: substantive quality counts more than visual polish. A beautifully designed page with mediocre text loses to an ugly page with excellent text. The AI looks straight through the design layer to the content underneath.

The pitfall of contextual collapse

On a well-built website, content exists in context. A product page mentions the product name in the header, so the body text can use pronouns: “It features advanced noise cancellation.” That works for a human who sees the whole page. It fails completely under AI extraction.

When the model pulls a passage from the middle of your page, it only gets that passage. If it says “It features advanced noise cancellation”, the model does not know what “it” refers to. The context did not travel along. I call this contextual collapse.

The solution is content in which every section and every paragraph stands on its own. The entity names, the product names, the explicit mention of what you are talking about: those have to recur in every logical unit. That feels repetitive when you read the page as a whole, but the AI never reads the whole. It reads loose chunks, and those have to be readable independently.

Multi-source synthesis: from position to share

Classic search gave you a list of sources. The user chose which to visit, and every source stood on its own. AI search synthesises across sources: a single paragraph of AI answer can combine a statistic from source A, an explanation from source B and an example from source C.

That creates a new competitive dynamic. In the past you fought for position, now you fight for share of the synthesis. Suppose an answer uses five sources: one source may deliver 60% of the content (the main source, heavily quoted), one 20%, and three others 5% each. The main source captures disproportionate value: brand visibility, potential click-through, authority. The small sources are practically invisible.

That is why selection rate alone is an incomplete measure. You also want to measure integration depth: how substantially does your content contribute when it is selected?

Platform differences

The architecture differs per platform, and that is relevant to your strategy:

In other words, Google rewards agreement and well-structured content with a clear hierarchy. ChatGPT rewards comprehensive, distinctive content that can serve as a primary source. “AI search” is therefore not one thing, and optimisation is best done with that variation in mind.

The grounding decision: not every question triggers retrieval

A nuance many people miss: not every question activates grounding. If a user asks something the model is certain about (basic facts, widely spread knowledge), it may answer from parametric knowledge alone. No search action, no retrieval, no chance for your content to be selected.

If someone asks something the model is less certain about (recent events, specialised topics, current information), it triggers grounding. It searches, retrieves and synthesises. That decision is made automatically, based on the model’s confidence.

Questions that usually trigger grounding:

• Recent events and current information
• Specialised or technical topics
• Requests for specific facts or figures
• Comparative questions (“best”, “top”, “recommended”)
• Local or time-sensitive information

Questions with low grounding:

• Established facts (“What is the capital of France?”)
• Conceptual explanations within the model’s training
• General knowledge that sits broadly in the training data

If your target questions fall into the low-grounding category, content optimisation has limited impact and influencing the training data itself is your only (much slower) path. If they fall into the high-grounding category, optimisation can have immediate effect: every question is a chance to be retrieved and selected. Because that decision can differ each time, AI search is inherently capricious. You can read more about that in my article on the probabilistic nature of AI answers.

The AI as the new first reader

For two decades we thought about two audiences: humans and search engines. AI search introduces a new primary reader: the AI itself. The AI reads, evaluates, extracts and synthesises your content before a human sees anything. If the AI does not select your content, the human never encounters it. The AI is the gatekeeper.

But the AI reads differently. It does not experience your design and does not simply count keywords: it understands meaning, assesses quality and judges usefulness, but stripped of all visual and emotional context. Content that leans on visual context, pronouns without an antecedent, or the assumption that the reader saw the whole page gets exposed. Content that communicates with precision and clarity is rewarded.

That is a higher bar than classic SEO asked for. In a strange way, the AI reader forces us to become better communicators, and whoever meets its demands usually meets the demands of human readers better too. If you want to weigh this against the old approach, read GEO vs SEO.

Frequently asked questions

What exactly is query fan-out?

Query fan-out is the mechanism where an AI search system splits your single question into several sub-questions and runs each of them separately against a search infrastructure. It then combines the results of all those sub-questions into one coherent answer. As a result, your content can surface for questions you never literally optimised for.

Does an AI model see my page the way a visitor does?

No. The model mainly sees text, not your design. Sometimes basic HTML structure survives, such as headings and bold text, but fonts, colours, images and layout disappear. That is why the substantive quality of your text weighs more heavily than the visual polish of your page.

What is contextual collapse and how do I avoid it?

Contextual collapse happens when the AI pulls a passage from the middle of your page without the surrounding context. Pronouns like “it” or “we” then become meaningless. You avoid it by making every section and paragraph readable on its own: repeat entity and product names and say explicitly what you are talking about.

Does every search query trigger retrieval in AI search?

No. For questions the model is certain about, it sometimes answers from its own parametric knowledge without retrieving anything. Only with uncertain, current or specialised questions does it trigger grounding and pull in external sources. Only on those grounded questions can your content compete to be selected.

Need help?

Want to translate this into execution? See how we approach this with AI visibility.