What AI engines can read decides what they can cite. See also: can AI crawlers read JavaScript-rendered pages, does llms.txt help you get cited by AI and should you block GPTBot in robots.txt.
What did research find about AI crawlers and JavaScript?
Research on AI crawlers and JavaScript found that the major AI crawlers download script files but never run them. In a December 2024 analysis, Vercel and MERJ reported that "OpenAI's GPTBot generated 569 million requests across Vercel's network in the past month", with Anthropic's Claude crawler at 370 million, and concluded that none of the major AI crawlers render JavaScript. The same study found that "Both crawlers spend significant time on JavaScript files (ChatGPT: 11.50%, Claude: 23.84%) despite not executing them." Googlebot is the exception: Google's JavaScript guide says that "Google Search runs JavaScript with an evergreen version of Chromium". A headless browser is a browser run by a program without a screen, and it is what rendering a page for a crawler requires.
| Crawler | Runs JavaScript? | What it sees on a client-rendered page |
|---|---|---|
| Googlebot | Yes, with evergreen Chromium | The rendered page, after a render queue |
| GPTBot and OAI-SearchBot (OpenAI) | No, per the 2024 study | The raw HTML only |
| ClaudeBot and Claude-SearchBot (Anthropic) | No, per the 2024 study | The raw HTML only |
| PerplexityBot | No, per the 2024 study | The raw HTML only |
Which sites are affected by AI crawlers skipping JavaScript?
Sites affected by AI crawlers skipping JavaScript are the ones whose main text arrives after the page loads. In the same 2024 data, 34.82% of ChatGPT's crawler fetches hit 404 pages, so AI crawlers already waste much of their budget before rendering even comes up.
- Single-page apps built with React, Vue or Angular that ship an empty
<div id="root">and fill it in the browser. - Site builders that load page text from an API after the page opens.
- Tabs, accordions and "load more" sections that fetch their content only when clicked.
Content that is in the HTML but hidden with CSS, such as a collapsed FAQ, is fine: the text is there for any crawler that reads the source. Server-rendered text is content present in the HTML before JavaScript runs, and it is the only text you can count on an AI crawler seeing.
How do you check whether AI crawlers can read your page?
Checking whether AI crawlers can read your page means looking at the raw HTML your server sends. Open the page, choose View Page Source (not Inspect), and search for a sentence from your main content. If it is not in the source, AI crawlers probably cannot see it. From a terminal, curl -s https://yoursite.com/page | grep "your sentence" runs the same test. Check robots.txt at the same time, because a blocked bot reads 0% of the page: OpenAI's crawler documentation says "OAI-SearchBot is used to surface websites in search results in ChatGPT's search features", and Anthropic's crawler article says "Claude-SearchBot navigates the web to improve search result quality for users."
How many websites need JavaScript to show their text?
More than a third of business websites need JavaScript to show their main text. In the GeoTest Benchmark Report 2026, 37.1% of the 4,999 sites had a homepage whose main text was missing without JavaScript. Client-side rendering is a way of building pages in which the browser runs JavaScript to draw the text after the page loads, and it is common on AI app builders: the main text needed JavaScript on 88.8% of the Lovable sites, 92.9% of the Base44 sites and 73.9% of the Replit sites in the sample.
What does it cost a site when AI crawlers cannot read it?
A site that AI crawlers cannot read loses its chance to be cited, and a citation is often all the visibility a search now gives. Pew Research Center data from 2025 shows users who saw a Google AI summary clicked a traditional result in 8% of visits, against 15% without one. G2's Answer Economy research, published in 2026, found that 71% of B2B software buyers rely on AI chatbots for software research, so a page those chatbots cannot read is missing from much of that research.
Do store and website builders have the same JavaScript problem?
Store and website builders have the JavaScript problem far less often than AI app builders. In the same benchmark, the main text needed JavaScript on 12.9% of Webflow sites, 9.1% of Squarespace sites, 21.2% of WooCommerce sites and 44.9% of Shopify sites. A Shopify store is worth checking template by template with View Page Source, since nearly half the Shopify sites in the sample needed JavaScript for their main text.
How do you fix a page AI crawlers cannot read?
Fixing a page AI crawlers cannot read means sending its text in the first HTML response, using one of three methods.
- Server-side rendering (Next.js, Nuxt, Remix, SvelteKit) builds the HTML on each request.
- Static prerendering writes an HTML file for each marketing page at build time.
- Dynamic rendering serves prerendered HTML to bots only. It works, but the bot and browser versions must match, or it looks like cloaking.
GeoTest checks this on every scan as "rendered parity" under Extractability (2 of the category's 8 points): whether the text a visitor sees is in the initial HTML. This page is prerendered the same way, so all of its text is in the HTML before any script runs. Google's guidance on AI features adds that a page must be indexed to appear as a supporting link, which also depends on crawlers reaching its text.
How does GeoTest measure whether AI crawlers can read a page?
GeoTest measures whether AI crawlers can read a page by comparing the words in the raw HTML with the words in the rendered page, a check called rendered parity. A page whose rendered text is at least 90% present in the raw HTML earns both points, and one at 60% or more earns 1. Extractability, the category this check sits in, was the strongest category in the benchmark at 90.1% of available points, so a page that fails it stands out.
