AI-ready website checklist: 25 technical points before launch

An AI-ready website is a site whose technical build lets AI crawlers fetch, read and understand the pages cleanly. It is not a visual trick, but a list of concrete build decisions: how you render, what you put in the HTML, which crawlers you allow and how fast you load. This article is the technical launch checklist, 25 points a developer works through before taking a B2B site live. It is about the build and the technology, not about your editorial GEO strategy. The two are complementary: without clean technology, no model will ever read your best content.

Why is the technical side decisive for AI findability?

Because an AI model can only cite what it can cleanly fetch and parse. Tools like ChatGPT, Perplexity and Google’s AI overviews send crawlers to your site, fetch the HTML and try to work out what your page is about. If that fetch fails, or if the bot receives an empty page that only gets built in the browser with JavaScript, you are out before your content has ever been assessed. In other words, the technology is the gateway: it decides whether you are in the running at all.

That makes this checklist a dev handover document. You use it separately from the editorial choices (which topics, which answers, which authority) you make in your GEO strategy. Here it is purely about one thing: are the foundations technically sound at handover? For the wider build process and the choices around it, see our guide to having a B2B website built.

Which points cover rendering and crawl access?

The most important rule: make sure your content is already in the first HTML response, not only after JavaScript has run. Many AI crawlers render JavaScript only partially or not at all, so anything that appears client-side only does not exist for them.

1. Server-side rendering or pre-rendering. Deliver the core content in the initial HTML, via SSR, static generation or pre-rendering. A purely client-side SPA whose text only appears after loading is risky for crawlers.
2. Important text sits in the HTML source. Use “view source” to check that your headings, paragraphs and links are visible without the browser running scripts.
3. A correct robots.txt. Do not block folders or resources crawlers need, and do allow the AI crawlers you want to give visibility to.
4. A deliberate choice per AI crawler. User agents such as GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot and Google-Extended (Gemini) can be allowed or blocked individually. If you want to appear in AI answers, do not accidentally block the bots you need for that.
5. An up-to-date XML sitemap. Generate it automatically, keep it clean (only indexable, canonical URLs) and reference it in robots.txt.
6. No unintended noindex or crawl blocks. A forgotten noindex tag or a staging block that travels along to production is a classic at launch. Check every template.
7. Correct HTTP status codes. Live pages return 200, removed pages 410 or 404, moved pages a 301. Soft 404s (a 200 on an empty page) confuse crawlers.

How do you make the structure machine-readable?

By using semantic HTML that makes the meaning of every element explicit, not just its styling. A model infers your page structure from your markup, so the clearer that markup is, the better it can parse your content.

8. One logical H1 per page that names the main question or the main topic.
9. A tidy heading hierarchy. H2s and H3s follow each other logically, without skipping levels. Phrase them as the questions a reader or an AI would ask.
10. Semantic elements such as header, nav, main, article, section and footer instead of a sea of generic divs.
11. Descriptive, unique title tags and meta descriptions per page that honestly summarise the content.
12. Alt text on meaningful images. It serves both accessibility and machine readability. Also read our most common accessibility mistakes.
13. Descriptive, stable URLs instead of cryptic parameters, plus a well-considered website architecture in which related pages hang together logically.
14. Internal links with descriptive anchor text. They show crawlers how your pages relate to each other and which ones matter most.
15. Answer-first pages. Put the direct answer at the top of a section and build out from there. That makes a passage easier to cite.

Which structured data belongs on the list?

Structured data (schema.org markup in JSON-LD) tells machines explicitly what a piece of content is: an organisation, an article, a frequently asked question, a product. It is no guarantee of visibility, but it removes guesswork and helps models interpret your entities correctly.

16. Organization markup with your company name, logo, location and contact details, so your brand is recognisable as an entity.
17. Article or BlogPosting markup on knowledge articles, with author and publication date.
18. BreadcrumbList markup that makes your site structure explicit.
19. FAQ or Q&A markup where you genuinely answer questions, not as a gimmick on every page.
20. Valid JSON-LD. Validate every template with a schema validator before launch. Dig into schema markup and the broader logic of structured data to pick the right types.

Which performance and technical points must be finished?

A fast, stable and secure site is not a luxury but a crawl condition: slow or error-prone pages get fetched less often and less deeply. Performance also hits your conversion directly, because a slow site costs you leads before a human has read anything.

21. Good Core Web Vitals. Keep load speed, interactivity and visual stability within healthy margins. Our explanations of Core Web Vitals and Google Lighthouse help you measure.
22. Mobile-friendly and responsive, because crawlers generally assess the mobile version.
23. HTTPS everywhere, without mixed-content warnings.
24. Canonical tags that point duplicates and parameter variants to a single preferred URL.
25. Clean, consistent redirects without chains or loops, certainly if you are migrating from an old site.

And what about llms.txt, is it needed?

Optional, and with realistic expectations. llms.txt is a proposed file in the root of your domain that would point models to your most important content, conceived in 2024. Adoption is low for now and no major AI provider officially leans on it heavily. You lose little by adding it, but do not count on it as a cornerstone. The gains sit in the 25 points above, not in an experimental text file.

Important to stay honest about: this checklist makes your site technically ready to be found, but guarantees no citations. Whether a model picks you also depends on your content, your authority and your relevance to the question. The technology opens the door, your content has to walk through it. We would rather steer on that full chain than on isolated scorecards.

The short summary

An AI-ready website starts at the build: deliver content in server-side HTML, give the right crawlers access via robots.txt, structure your pages semantically, add valid schema markup and keep your performance and redirects clean. Work through these 25 points as a technical launch checklist, separate from your content strategy, and you go live machine-readable. Alongside the technology, also nail down what belongs in your website contract and SLA, so handover, ownership and aftercare are locked down legally too. Want an experienced team watching over these foundations with you? See how we can build your website or book your free intake.