GeoTest
Product updateFull scans

Full scans now read your sitemap first and skip noindex pages

· October 28, 2026 · ◷ 4 min read · Updated October 29, 2026

A full GeoTest scan now starts from your sitemap, the list of pages you actually want found, and leaves out any page marked noindex. Your site score now reflects the pages you publish on purpose, not whatever a link crawl happened to reach. Both changes are live for every full scan since 26 October 2026.

Here is a small confession. Until this week, a full scan picked its pages the way a tourist picks restaurants: it followed links and hoped for the best. Most of the time that worked. Some of the time it scored a thank-you page, a staging copy of the contact form, or an article you deleted in 2024 that one old link still pointed to. None of those are pages you want an AI engine to cite, so none of them should move your score.

What changed in full scans?

Full scans changed in two ways on 26 October 2026: they read your sitemap first, and they leave out pages marked noindex.

  • Your sitemap comes first. When your sitemap lists your site, a full scan scores those pages: your homepage and main pages first, then your newest articles by their last-modified date. Pages your sitemap lists that no longer exist are skipped instead of scored.
  • Noindex pages are left out. If a page tells search engines not to index it, a full scan now leaves it out of your site score and says so in the scan timeline.
  • Nothing breaks if you have no sitemap. Sites without a usable sitemap are crawled through their links, exactly as before.

Why does the sitemap matter for a GEO Score?

The sitemap matters for a GEO Score because it is the one list on your site that says "these are the pages I want found". A sitemap is an XML file, usually at /sitemap.xml, that lists the URLs you want search engines to crawl, often with the date each one last changed. Google and Bing both read sitemaps to discover pages, and Google's AI Overviews draw on the same Google index. Google's guidance on AI features says “The best practices for SEO remain relevant for AI features in Google Search (such as AI Overviews and AI Mode).” Scoring the pages in it means your score describes the site you present to the world, not the corners a crawler wandered into.

It also changes which pages get scored on bigger sites. A full scan covers up to 200 pages. When your sitemap lists more than that, the scan keeps your homepage and main pages and saves at least 40 places for your newest articles, so a site with years of posts is scored on what it publishes now.

How does a full scan find your sitemap?

A full scan finds your sitemap the way most crawlers do: it reads the Sitemap: lines in your robots.txt first, then tries common paths such as /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml and /sitemaps.xml. The first real sitemap it finds is the one it uses.

A file counts as a real sitemap only if it contains a <urlset> or <sitemapindex> element, so an HTML error page served with status 200 does not count. If the sitemap is an index, the scan fetches its child sitemaps, up to two levels deep, and extracts every <loc> URL they list.

A Sitemap: line in robots.txt is worth adding even if your sitemap sits at /sitemap.xml. Google and Bing both read a Sitemap: https://yourdomain.com/sitemap.xml line in robots.txt, and so does a full scan. Changes to robots.txt are not picked up everywhere at once: OpenAI's crawler documentation states that “For search results, please note it can take ~24 hours from a site’s robots.txt update for our systems to adjust”.

What did it change on a real site?

On specloop.ai, our sister platform, the old link crawl found 61 pages on 22 October, 2 of them marked noindex, and scored the site 78. The first sitemap-led scan on 26 October read 57 pages from the sitemap, scanned all 57 with 0 failures in 63 seconds, and scored 79. The difference is small because specloop.ai is tidy. On a site with a cluttered link graph, old campaign pages or a staging copy, it will be bigger.

The noindex rule was checked the same way: a 13-page site with a noindex contact page was scored on 13 pages before the change and on 12 after it, with the timeline saying "1 page is marked noindex and was left out of the score".

What does noindex mean, and when is a noindex page still scored?

A noindex tag is a robots meta tag that asks search engines not to show a page in their results, and a full scan now treats it as your instruction. Two exceptions keep the report honest:

  • The page you submit is always scored. If you scan a noindex page on purpose, you get its score.
  • A site that is noindex everywhere is scored as it is. A staging copy, for example. Leaving every page out would leave nothing to score, so the scan scores the pages and the "Not noindex" check explains why the number is low.

Noindex is sent per URL, either in the page's HTML as <meta name="robots" content="noindex"> or in an HTTP response header as X-Robots-Tag: noindex. A full scan reads the meta tag in each page's HTML when it decides which pages to leave out.

Robots.txt is a different tool: a Disallow rule controls crawling, not indexing, while noindex lives on the page itself or in its response headers. Google's introduction to robots.txt explains the file's rules and limits. Leaving noindex pages out of the score also matches how AI search works. Google's guidance on AI features, updated in 2025, states that “To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed”, so a page marked noindex cannot appear as a supporting link there.

Quick scans are unchanged: they score the one page you give them.

What makes a good sitemap?

A good sitemap lists only the pages you want found, as absolute canonical URLs, with honest dates. A sitemap is a discovery and canonicalization signal, not permission to index, a quality judgement or a guarantee of indexing. Search engines still decide which listed pages to crawl and index.

  • Absolute, canonical URLs. A sitemap should contain absolute canonical URLs on one host and one protocol, and stay within the published size limits: the sitemaps.org protocol allows 50,000 URLs and 50MB uncompressed per file.
  • An index for big sites. A sitemap index lists other sitemap files, and it is used when one file would exceed the size cap.
  • Honest lastmod dates. A full scan picks your newest articles by their lastmod date, so lastmod should be an honest date of meaningful change, not the build time stamped on every URL.
  • No noindex pages. A URL that is listed in your sitemap and also sends noindex gives search engines a mixed signal, so either the noindex or the sitemap entry should be removed.

What should you do now?

Two checks are worth doing now, and both take a few minutes. First, open yoursite.com/sitemap.xml and make sure it lists the pages you care about, not just a handful, and that the deleted ones are gone. Hosted builders such as Shopify, Wix, Squarespace, Webflow and Ghost create a sitemap automatically; hand-built sites and many frameworks do not. If yours lists fewer than 5 public pages, GeoTest falls back to the link crawl. Then run a fresh full scan from your dashboard: the timeline will say whether it read your sitemap, how many pages it found there, and which pages it left out as noindex.

The full list of changes is on the changelog (releases 1.85 and 1.86).

Frequently asked questions

Short answers to common questions about the topics in this post.

Does a full GeoTest scan use my sitemap?

Yes, since 26 October 2026 a full GeoTest scan uses your sitemap first when it lists at least 5 public pages, scoring your homepage and main pages first and then your newest articles. Sites without a usable sitemap are crawled through their links as before.

Are noindex pages counted in my GeoTest score?

No, noindex pages are not counted in a full scan's GeoTest score. Pages with a noindex robots meta tag are left out of the site score and listed in the scan timeline. The page you submit is always scored, and a site that is noindex everywhere, such as a staging copy, is scored as it is.

What happens to pages my sitemap lists that no longer exist?

Pages your sitemap lists that no longer exist are skipped. A page that answers with an error, such as a 404, is recorded as failed instead of scored, so a stale sitemap does not pull your score down.

Do quick scans work differently now?

No, quick scans work exactly as before: a quick scan still scores the one page you submit.

Quick answers on this topic

All answers →
Cover of the AI Search Readiness Checklist
Free download · Checklist · PDF
AI Search Readiness Checklist

The checks an answer engine runs before it cites a page, as a list you can work through on your own site.

The GEO Report

The GEO Report. Weekly: what changed in AI search and what to fix. Written by Jason Gordon.

One email a week. Unsubscribe any time.

Get the fix for your page

Your GEO Score in 15 seconds, every issue priced in points. Paid plans write the fix from your own content. No signup to scan.

AI search in figures

These AI search figures show how fast answers are replacing clicks, and each one is dated and sourced. Generative Engine Optimization (GEO) is a method for structuring a website so AI answer engines can read, trust and cite it. A citation is defined as a link to your page inside an AI answer, and the GEO Score refers to a 0 to 100 rating of how ready a page is to earn one. These published figures show why it matters; the reasoning behind every weight is in the methodology.

How much do AI summaries reduce clicks?

AI summaries cut clicks by about half: Pew Research Center found searchers clicked a result 8% of the time when an AI summary appeared, versus 15% without one, a 47% drop (July 2025).

How common are zero-click searches in 2025?

Zero-click searches rose from 56% to 69% between May 2024 and May 2025, and 83% of searches end without a click when AI Overviews show, according to Similarweb (2025).

What share of websites are ready for AI search?

Very few websites are ready: only 2% of the 537 sites in the GeoTest benchmark scored 80 or more in September 2026, and the median score was 47. Those scores came from the rubrics in use before GEO v6.0.

How much search traffic are publishers losing?

Publishers are losing about a third of their Google referrals: across 2,500+ publisher websites, Google search referral traffic was down 33% globally and 38% in the US year over year as of early 2026 (Chartbeat).

How many buyers research with AI chatbots in 2026?

71% of B2B software buyers rely on AI chatbots for software research (G2, 2026), which is why being cited matters as much as ranking. See what GEO is.

FindingFigureSourceDate
Searchers who click a result when an AI summary appears8%, versus 15% without one (a 47% drop)Pew Research CenterJuly 2025
Zero-click searches, May 2024 to May 202556% rising to 69%Similarweb2025
Searches ending without a click when AI Overviews show83%, and 93% in AI ModeSimilarweb2025
Google referral traffic to 2,500+ publishers, year over yeardown 33% globally, 38% in the USChartbeatearly 2026
B2B software buyers relying on AI chatbots for software research71%G2, The Answer EconomyMarch 2026
Visibility lift from adding attributed quotations to a page41%GEO study (Aggarwal et al.)2023
Sites in the GeoTest benchmark scoring 80 or more, scored under the rubrics in use before GEO v6.02% of 537 sitesGeoTest benchmarkSeptember 2026

Find GeoTest on

  • Featured on Altern
  • Featured on Aura++
  • Better Launch
  • GeoTest on DodoLaunch
  • Fazier badge
  • Featured on FoundrList
  • Featured on Launch
  • GeoTest on Nick Launches
  • GeoTest, featured on Smol Launch
  • GeoTest - Featured on Startup Fame
  • Launched on StartupBase
  • Featured on Twelve Tools