On 21 October 2026, Google updated its guide to controlling what you share with Google Search.
Google did not launch any new controls. Instead, it brought the available options together on one page, covering everything from removing a page to controlling whether your content appears in AI Overviews and AI Mode.
This matters because indexing problems often happen when websites use the wrong control for the job.
For example, using robots.txt when you need noindex, blocking a page in robots.txt before Google can see its noindex tag, or using nosnippet to control AI visibility without realising that it also removes normal search snippets.
Another common mistake is treating a temporary removal request as a permanent solution.
The control you choose determines what Google can crawl, index, display and use in its search features. Get it wrong, and you could lose organic traffic, prevent important pages from appearing in Search or accidentally expose information you thought was hidden.
This guide explains what each control does, what it does not do, and how to use the right option for different types of content.
We’ll also cover the mistakes that SEO, content, marketing and development teams need to watch for, along with a Python script to help check the indexing rules applied to your URLs.
1. What Has Changed?
Google’s Search Central changelog records an update to its Control what you share with Google Search documentation on 21 October 2026.
The updated guide brings together several existing controls:
- The robots
metatag. - The
data-nosnippetattribute. - The
X-Robots-TagHTTP header. - The Search generative AI control.
Google says the aim is to keep the documentation up to date and reflect the options currently available for Google Search.
The guide also explains why a website might want to keep certain content out of Search. There are three main reasons.
Protect information intended only for people who already have access. For example, private documents or content behind a login should not be publicly accessible through Google Search.
Keep low-quality content out of search results. This includes spammy user-generated content and pages that offer little value.
Help Google focus on important pages on large websites. Removing unnecessary duplicates and managing low-value URLs can help Google concentrate its crawling on pages that matter.
One important addition to the documentation is the Search generative AI control.
According to the source material, Google launched this Search Console control in the UK on 3 June 2026 following a requirement from the Competition and Markets Authority (CMA). Google said it had reached all websites by 31 August 2026.
The control now appears alongside familiar options such as noindex and password protection. It is an official part of the tools available to manage how content appears in Google’s search features.
The Seven Main Controls Google Lists
Each control serves a different purpose. Some prevent content from being accessed, others remove content from Search, and some limit how Google displays or uses it.
| Control | Applies to | What it does | What it does not do |
|---|---|---|---|
| Remove the content | Any content | Removes the content from your website and other places where it is hosted, if those copies are also removed. | Automatically remove copies hosted by other websites. |
| Password protection | Any content | Prevents people and crawlers without access from viewing the content. Already-indexed content should eventually disappear from Search. | Prevent a login page from revealing details through its URL or page title. |
noindex meta tag or X-Robots-Tag header | Pages and files | Tells Google to remove a page or file from Search after it has crawled the URL and seen the rule. | Work if robots.txt blocks Google from crawling the URL and seeing the rule. |
robots.txt disallow | Primarily relevant here to images and video | Prevents Google from crawling blocked media. | Guarantee that a web page will stay out of Search. Google can still index a blocked URL from links to it. |
Robots meta tag, data-nosnippet and X-Robots-Tag display rules | Pages, files and parts of pages | Control snippets, previews, translation and when content becomes unavailable. | Remove a page from Search unless the rule specifically uses noindex. |
| Search generative AI control | A website or Search Console property and its child properties | Includes or excludes content from AI Overviews, AI Mode and generative AI features in Discover. | Change rankings across ordinary Search or control AI training. |
| Google properties opt-out | The whole domain | Removes crawled content from Google Shopping, Flights, Hotels, vacation rentals and local search results, within 30 days. | Target just one subdomain, directory or individual page. |
Two other options are worth knowing about.
The Removals tool can hide a page from Google Search in about a day, but the removal generally lasts for around six months. It is a temporary measure, not a permanent solution.
Google’s guidance on redacted information explains how private information can still leak from documents, images and URLs, even when you believe it has been hidden.
Important Details That Are Easy to Miss
Google’s documentation contains several rules that can have a significant effect on indexing and search visibility.
nosnippet also affects AI Overviews and AI Mode.
Google’s robots meta tag documentation says that nosnippet prevents content from being used as direct input for AI Overviews and AI Mode. However, it also removes ordinary text snippets from search results.
The max-snippet rule can limit how much text is available for snippets and direct input.
Robots meta tags in the page body still count.
Google respects robots meta tags in the HTML body as well as in the head. A tag accidentally added by a plugin, widget or page component can therefore affect indexing.
Google applies the more restrictive rule when rules conflict.
When several rules apply, Google follows the more restrictive instruction. If both general robots and Google-specific googlebot rules are present, Google combines their negative rules.
Only certain crawler names are recognised in robots meta tags.
The documentation specifically recognises googlebot and googlebot-news. A tag addressed to an unrecognised crawler name is ignored.
A noindex rule intended for robots also does not automatically cover non-search crawlers such as AdsBot.
data-nosnippet only works on span, div and section elements.
If the element is not closed correctly, content after it can also be hidden from snippets. Adding or removing the attribute with JavaScript can make the result unpredictable.
Structured data can still be used.
The rules described here do not generally prevent Google from using structured data, except for description values. Structured data inside a data-nosnippet block can still be used, so sensitive information must be removed from the structured data separately if necessary.
unavailable_after can remove content after a specified date.
This is useful for time-sensitive content, such as an expired offer or event. Google also reduces how often it crawls the page after the date has passed.
noimageindex has a limited scope.
Adding noimageindex to a page prevents images from being indexed through that page. The same image may still be indexed through another page.
To prevent an image from being indexed anywhere on Google, use an X-Robots-Tag: noindex header on the image itself.
2. Why Do These Controls Matter?
Each control is relatively straightforward when used on its own. Problems arise when the wrong option is chosen, two rules conflict or a temporary fix is mistaken for a permanent solution.
Here are eight common mistakes to watch for.
1. Using robots.txt to Keep Pages Out of Search
Google’s documentation is clear: blocking a URL in robots.txt is not a reliable way to remove a web page from Search.
A blocked page can still be indexed if Google discovers its URL through links. It may then appear in search results without a description because Google cannot crawl the page to read its content.
There is another problem. If Google cannot crawl the page, it cannot see a noindex tag on it either.
Use noindex when your goal is to remove a page from Search, and make sure Google can crawl the page to see the rule.
Our guide to robots.txt rules Google ignores explains how Google handles these rules.
2. Using noindex When Google Cannot See It
The noindex rule only works when Google can crawl the URL and read the instruction.
If a page is blocked by robots.txt, sits behind a login, is rejected by a firewall or returns an error instead of the page, Google may never see the tag.
The page can therefore remain in Search despite your attempts to remove it.
Google also warns that pages it rarely crawls can take months to be revisited. If a page needs to be removed promptly, you may need to request a recrawl after making the change.
Our analysis of Googlebot being blocked with 403 errors explains how firewalls can prevent Google from accessing pages.
3. Using AI Controls Without Understanding the Trade-Off
There are three main levels of control when you want to limit how your content appears in AI search features.
| Level | Control | What you give up |
|---|---|---|
| Whole site or property | Search generative AI control | AI Overview, AI Mode and Discover AI appearances and their links across the selected site or property. |
| Individual page | nosnippet or max-snippet | nosnippet removes ordinary text snippets as well; max-snippet limits the length of the text Google can use. |
| Part of a page | data-nosnippet | Text from the marked section is excluded from snippets. Google’s AI feature documentation lists this among the ways to limit what is displayed but does not explain exactly how it applies to AI answers. |
Excluding content from AI features can also mean losing links and citations that those features might otherwise provide.
For information-led websites, this is a commercial decision as well as a technical one. Our research into the answer capture ratio found that click-through rates can fall sharply for question-based searches.
Before blocking content from AI search, consider whether the potential loss of visibility and referrals outweighs the benefit of restricting its use.
4. Treating Temporary Removals as Permanent
Google’s Removals tool can hide a page from Search in about a day, but the request generally lasts for around six months.
It also applies only to the URL variations you submit. Differences in case, parameters or protocol may require separate requests.
If the page is still live and indexable when the temporary removal expires, it can return to Search.
Every temporary removal should therefore have a permanent fix behind it, such as deleting the content, protecting it with a login or applying noindex.
5. Publishing Information You Thought Was Hidden
Documents can reveal information even when the sensitive text appears to have been removed.
Google’s guidance highlights several risks:
- Drawing black boxes over text instead of properly removing the text.
- Leaving tracked changes or document history in the file.
- Keeping uncropped images inside documents.
- Leaving editors’ names or other identifying details in file metadata.
- Allowing information to be recognised from images.
- Including names or email addresses in URLs.
A URL can be indexed even if robots.txt blocks the page. Blocking the URL does not make the information private.
For UK businesses, a leak involving personal or confidential information can become a data protection issue as well as an SEO problem.
6. Allowing Low-Value Content to Build Up
Google warns that indexing low-quality user-generated content may negatively affect a site’s rankings. Its spam policies also tell websites hosting scaled content of little value to keep that content out of Search.
This matters for forums, marketplaces, directories and websites with large numbers of user-created pages.
Our scaled content abuse experiment demonstrates how quickly large volumes of low-value pages can build up.
Rather than waiting for these pages to become a problem, set clear rules for when new content is allowed into Search.
7. Applying an Opt-Out More Widely Than Intended
The opt-out for Google properties applies to the whole domain and its subdomains. It cannot be limited to a single directory.
It can also remove crawled content from Google Shopping and local search results, among other services.
A retailer trying to solve one visibility issue could therefore lose visibility across several Google services.
Before using a domain-wide control, check which pages, subdomains and Google services it will affect.
8. Overlooking Rules Added by Plugins and Templates
Google can apply robots meta tags placed in the page body. It also follows the most restrictive applicable rules and combines general rules with Google-specific googlebot rules.
This means a rule added by a component, plugin, staging configuration or template can affect many pages without anyone editing them individually.
For example, a promotional component might inject noindex into a page body. Google can still apply that instruction, potentially removing the page from Search.
An indexing audit should therefore check the HTML Google receives, not just the settings visible in the content management system.
Common Mistakes and Their Consequences
| Mistake | Typical cause | Consequence |
|---|---|---|
Using robots.txt on web pages | Confusing crawling with indexing | URLs can remain indexed, while Google cannot see a noindex rule. |
noindex that Google cannot see | Robots blocks, login pages, firewalls or errors | Pages may stay in Search. |
| Broad AI exclusion | Using nosnippet or a site-wide control for a narrow problem | Lost snippets, AI citations and links. |
| Temporary removals with no permanent fix | Using the Removals tool on its own | Content can return after about six months. |
| Documents that reveal hidden information | Poor redaction, metadata or identifying details in URLs | Sensitive information can remain findable in Search. |
| Domain-wide opt-outs | Applying a property-level setting to solve a local problem | Lost visibility in Shopping or local results. |
| Hidden indexing rules | Body tags, plugins or conflicting instructions | Templates can be removed from Search without anyone noticing. |
3. Which Businesses Are Affected?
Almost every website uses at least some of these controls. The main challenge is that different people often set them at different times, using different systems and with different objectives.
Publishers and Information Websites
Publishers need to decide how much content they want Google to show in AI features.
Options include the site-wide Search generative AI control, page-level nosnippet or max-snippet, and section-level data-nosnippet for selected passages, such as licensed or premium content.
The decision should reflect the commercial value of appearing in search results and being cited in AI answers.
E-commerce Websites
Retailers need to manage faceted navigation, internal search pages, basket pages and other URLs that may not belong in Search.
They also need to understand the impact of domain-wide Google properties opt-outs, which can affect Google Shopping.
The key is to decide which page types should be indexed and implement the rules consistently across templates.
Travel and Hospitality Businesses
Google’s property opt-out covers services such as Hotels, Flights and vacation rentals.
Because the setting applies to the whole domain, businesses need to understand its wider effects before using it to solve a specific problem.
Local and Multi-Location Businesses
The opt-out for local search results applies globally and covers every subdomain.
Businesses with separate websites or subdomains for different locations should check the scope of any domain-level setting before applying it.
Healthcare, Finance, Legal and Public Sector Organisations
These organisations often publish PDFs, reports and forms that may contain confidential information or personal details.
Document metadata, tracked changes, images and URLs can all expose information that was meant to be hidden.
For these organisations, the right controls and a reliable publishing process are particularly important.
Forums, Marketplaces and User-Generated Content Websites
Websites that accept user submissions need a clear policy for keeping low-quality and spammy content out of Search.
New profiles, empty listings and unmoderated submissions should not automatically become indexable simply because they have been published.
Large Websites with Staging, Archive and Duplicate Content
Large sites may have multiple environments, templates, archives and URL variations.
Each page type needs a clear indexing rule. These rules must also be applied consistently so that changes to one template do not accidentally remove an entire section from Search.
4. What Should Businesses Do?
Treat indexing controls as a site-wide policy rather than a collection of one-off fixes.
First, decide what each type of page should do in Search. Then choose the control that achieves that goal, implement it in templates or server settings wherever possible, and test it before releasing changes.
The following steps explain how to organise the work.
Step 1. Choose the Right Control for Each Goal
Who: SEO lead
Start by defining the outcome you need. Then choose the control that matches it.
| What you want | Use | Do not use |
|---|---|---|
| A page should never appear in Search | noindex, while allowing Google to crawl the page | robots.txt |
| Private content should be inaccessible to people without permission | Password protection or a login, with the login page set to noindex where appropriate | noindex alone, which does not make public content private |
| A PDF, image or other file should be removed from Search | X-Robots-Tag: noindex on the file | A robots meta tag, which files cannot carry |
| Images should stay out of Google Images | robots.txt rules for Googlebot-Image, or X-Robots-Tag: noindex on the images | Both controls together without a clear reason; Google’s guidance says using both in this way is unnecessary |
| A page needs to be hidden from Search quickly | The Removals tool, followed by a permanent fix | The Removals tool on its own |
| Google should spend less time crawling duplicates and low-value URLs | robots.txt, where the pages are not already indexed and crawling needs to be restricted | robots.txt as a way to remove already-indexed pages |
| Part of a page should not appear in snippets | data-nosnippet on a span, div or section | Hiding text with CSS or changing the attribute unpredictably with JavaScript |
| Text should not be used as direct input for AI Overviews and AI Mode | nosnippet, accepting that ordinary snippets are also removed, or max-snippet to limit text length | noindex, unless you also want to remove the page from Search |
| The whole site should be excluded from AI Overviews, AI Mode and Discover AI features | Search generative AI control in Search Console | Blocking Googlebot, which can remove the site from ordinary Search |
| Content should not be used to train or ground Gemini outside Search | The Google-Extended robots.txt token | The Search generative AI control, which does not cover training |
| Content should be removed from Google Shopping, Flights, Hotels or local results | The property opt-out in Search Console, applied to the whole domain | Expecting the setting to affect just one directory |
| A page should disappear after a particular date | unavailable_after | Leaving expired offers or event pages live and indexable indefinitely |
The most important point is to match the rule to the intended outcome. Preventing crawling, preventing indexing and limiting how Google displays content are three different things.
Step 2. Implement Each Control Properly
Who: Developers
Where possible, implement indexing rules in templates and server configuration rather than individual page editors.
This makes behaviour more consistent and reduces the risk of accidental changes.
Use noindex on Pages
Add the following meta tag to the page’s HTML:
<meta name="robots" content="noindex">Use googlebot instead of robots only when the rule needs to apply specifically to Google.
Then check three things:
- The page is not blocked by
robots.txt. - It returns a successful HTTP status, normally
200, to Googlebot. - A firewall or security system is not preventing Google from accessing the page.
If Google cannot crawl the page, it may never see the noindex instruction.
Use X-Robots-Tag for Files
Files such as PDFs cannot carry an HTML robots meta tag. You can control their indexing with an HTTP response header instead.
For example, the source provides these Apache and NGINX examples:
# Apache (.htaccess or httpd.conf)
<Files ~ "\.pdf$">
Header set X-Robots-Tag "noindex"
</Files>
# NGINX
location ~* \.pdf$ {
add_header X-Robots-Tag "noindex";
}These examples show how a noindex header can be applied to PDF files. Test the configuration in your own environment before deploying it, and check that it applies to the intended URLs.
Use data-nosnippet for Specific Sections
Use this attribute when a particular part of a page should not appear in search snippets.
<section data-nosnippet>
<p>Trade prices for account holders.</p>
</section>Google honours this attribute on span, div and section elements.
Make sure the element opens and closes correctly. An unclosed element can cause everything after it to be excluded from snippets.
Avoid adding or removing the attribute with JavaScript, as this can make the result unpredictable.
Remember that structured data inside a data-nosnippet section may still be used. If a value must not be shown, check and remove it from the relevant structured data too.
Make Temporary Removals Permanent
When you submit a removal request, include all relevant URL variations, including differences in capitalisation, parameters and protocol.
Then implement a permanent fix before the temporary removal expires. Depending on the situation, this could mean deleting the content, protecting it with a password or applying noindex.
Step 3. Make a Clear Decision About AI Visibility
Who: Leadership, marketing and legal
Controlling AI visibility can affect how people discover your content, so the decision should not be left to developers alone.
Use this process:
- Decide whether AI visibility is valuable to your business. Consider whether being cited in AI Overviews and AI Mode could bring useful exposure or traffic.
- Choose the narrowest control that meets your needs. If only selected content needs to be excluded, consider
data-nosnippetormax-snippet. Usenosnippetonly where losing ordinary search snippets is acceptable. - Use the site-wide control only when appropriate. Search Console’s Search generative AI control can exclude a whole site or property. Child properties can have different settings from a parent, allowing some sections to be treated differently.
- Manage training preferences separately. The
Google-Extendedtoken is separate from the Search generative AI control. - Measure the effect. Compare impressions, clicks and AI-related referrals for the pages affected by the change.
The decision should be based on the value of the content, the reason for restricting it and the likely effect on search visibility.
Step 4. Publish Documents Safely
Who: Content and compliance teams
Before publishing reports, forms, PDFs or other documents, check that they do not reveal information that should remain private.
Use the following checklist:
- Use proper redaction tools that remove text rather than placing black boxes over it.
- Crop and flatten images before including them in documents. Export them to a suitable format, such as PNG or WebP.
- Remove document metadata and tracked changes. Export to a format that does not retain unwanted change history.
- Keep names and email addresses out of URLs and filenames where possible. Use non-identifying identifiers instead.
- If sensitive information is exposed, take the content down, use the Removals tool where appropriate, publish a corrected version and ask other hosts to remove their copies.
- Verify every website you control in Search Console so you can request removals quickly when necessary.
Prevention is better than relying on emergency removal requests after information has already been published.
Step 5. Keep Low-Value and User-Generated Content Out of Search
Who: Content and community teams
Websites with user-generated content need rules that determine when a page is ready to be indexed.
For example:
- Keep new or unmoderated submissions, thin profiles and empty listings on
noindexuntil they meet your quality requirements. - Allow indexing once the content provides enough value and has passed the appropriate checks.
- Add
rel="ugc"orrel="nofollow"to links in user-generated content where appropriate. - Moderate spam quickly.
- Exclude scaled pages that offer little value from Search, in line with Google’s spam policies.
These rules should be built into the publishing process rather than applied manually after problems appear.
Step 6. Manage Indexing Rules Across the Whole Site
Who: SEO lead and developers
Create a central record of how each type of page should behave in Search.
This should cover every important template, file type and URL pattern.
1. Keep an indexing register.
Create one row for each template or URL pattern. Record whether it should be indexable, set to noindex, blocked from crawling or password-protected.
Include any snippet or AI-related rules, as well as the person responsible for the setting.
2. Put rules in templates and server configuration.
Avoid relying on individual page editors wherever possible. Templates and server settings make it easier to apply the same rules consistently.
3. Test each release.
Use the auditing script described below to check URLs that should remain indexable and those that should not.
4. Monitor Search Console.
The Page indexing report helps you investigate pages excluded by noindex. URL Inspection lets you check what Google received when it crawled a page.
5. Review settings regularly.
Review domain-wide opt-outs and AI controls every quarter. Record who changed each setting and why.
Step 7. Give Each Team Clear Responsibilities
| Team | What changes | First actions |
|---|---|---|
| Content | Exclusions should follow the agreed register, not ad hoc page-level changes. | Stop using robots.txt to try to hide web pages. Follow the document redaction checklist. |
| Marketing | Owns the AI visibility decision with leadership and legal. | Agree the AI policy from Step 3 and measure its effects. |
| Development | Implements indexing controls in templates and server headers, and tests them before release. | Add X-Robots-Tag rules for files where required, remove accidental body meta tags and add the script to the release process. |
| Legal and compliance | Owns sensitive information, document redaction and removal requests. | Approve the publishing checklist and ensure Search Console access is available for urgent removals. |
The aim is to make indexing rules part of normal website management, not something the team discovers only after a page disappears from Google.
5. Developer Toolkit: index_controls_audit.py
The index_controls_audit.py script checks URLs and reports the indexing and display rules it finds.
It uses Python’s standard library and relies on the robots_check.py script from our robots.txt guide to check robots.txt. Both files need to be in the same folder.
The audit checks several things:
- Robots meta tags in both the head and body of a page.
- General
robotsrules and Google-specificgooglebotrules. X-Robots-TagHTTP headers, including headers addressed to a named crawler.- How multiple rules combine, including situations where the most restrictive rule wins.
- Whether the page is indexable, whether snippets are allowed and whether the text can be used as direct input for AI Overviews and AI Mode.
- Body-placed meta tags, unsupported
data-nosnippetelements, unsupported rules such asnoai, and expiredunavailable_afterdates. - URLs where Google may be unable to see the indexing rules because
robots.txtblocks crawling.
The script also supports --expect-noindex and --expect-index options. These let you check whether URLs meet your expected rules and return an error when they do not. This means you can include the audit in your website release checks.
Example: Running the Audit
The following example checks five URLs on a test website:
python3 index_controls_audit.py \
https://www.example.co.uk/products/drill \
https://www.example.co.uk/search?q=drill \
https://www.example.co.uk/private/report \
https://www.example.co.uk/guides/radiator \
https://www.example.co.uk/files/price-list.pdf \
--expect-noindex "/search|/private/" \
--expect-index "/products/|/guides/"Example output:
https://www.example.co.uk/products/drill
INDEXABLE, snippets and AI use allowed
(site-wide AI control not visible here)
https://www.example.co.uk/search?q=drill
INDEXABLE, snippets and AI use allowed
(site-wide AI control not visible here)
FAIL expected noindex (matches /search|/private/)
https://www.example.co.uk/private/report
BLOCKED by robots.txt (rules unseen)
ERROR robots.txt blocks Googlebot (disallow: /private/):
Google cannot see any rule below, and the URL can still be indexed from links
https://www.example.co.uk/guides/radiator
NOINDEX
WARN <meta name='robots' content='noindex'> is in the body.
Google still applies it; check it is meant to be there
(components and widgets often inject these)
FAIL expected indexable (matches /products/|/guides/)
https://www.example.co.uk/files/price-list.pdf
INDEXABLE, snippets and AI use allowed
(site-wide AI control not visible here)
INFO Non-HTML (application/pdf) with no X-Robots-Tag:
indexable if Google supports the format
5 URL(s), 2 expectation failure(s)Three of the five URLs illustrate potential problems.
The internal search page is indexable when it should not be. The script flags that the URL does not meet the expected noindex rule.
The private report is blocked by robots.txt. Google may be unable to see any noindex instruction on the page, and the URL could still be indexed from links.
The guide has an unexpected noindex tag in its body. A component or widget may have added it, preventing the guide from being indexed when it should be available in Search.
The price list PDF is also worth checking. It has no X-Robots-Tag header, so the audit reports that it may be indexable if Google supports the file format. That is only a problem if the file was meant to stay out of Search.
The example is designed to demonstrate the checks. It is not evidence from a live website.
The Main Rule-Combining Function
The following excerpt shows part of the script’s logic for combining robots directives.
This is an excerpt, not a complete, standalone function. The full implementation needs the surrounding definitions and also handles other rules.
def combine(rules, now=None):
"""Most restrictive wins. Returns a summary dict plus notes."""
for name, value in rules:
if name in ("noindex", "none"):
eff["index"] = False
if name in ("nofollow", "none"):
eff["follow"] = False
if name == "nosnippet":
eff["snippet"] = False
elif name == "max-snippet":
n = int(value)
if n == 0:
eff["snippet"] = False
elif n > 0:
eff["max_snippet"] = (
n if eff["max_snippet"] is None
else min(eff["max_snippet"], n)
)
elif name == "unavailable_after":
when = parse_date(value)
if when is not None and when <= now:
eff["index"], eff["expired"] = False, True
# The full function also handles image previews,
# translation and unknown rules.The script has two important limitations.
It cannot see Search Console settings. The Search generative AI control and Google properties opt-out are managed in Search Console, so this page-level audit cannot determine those settings from the HTML alone.
It checks what your own machine receives. A firewall might treat Googlebot differently from an ordinary browser. For that reason, compare the audit with URL Inspection in Search Console when checking important pages.
Use the script as a release check and an aid to investigation. It helps identify unexpected rules, but it does not replace checks of Google’s actual access to a URL.
6. What We’re Watching Next
Several developments could change how websites manage indexing and AI search visibility.
More Precise AI Controls
The Search generative AI control works at site or property level. The page-level options discussed here, such as nosnippet and max-snippet, also affect ordinary search snippets.
The source material says the CMA gave Google nine months from 3 June 2026 to implement its publisher requirements. Based on that timetable, further developments were expected before early March 2027.
More precise controls would help website owners manage how specific content appears in AI features without unnecessarily losing visibility elsewhere in Search.
Google’s Documentation May Become Clearer
The updated control page now lists the Search generative AI control, but the source notes that Google’s separate guide to AI features on websites did not yet mention it.
The documentation also did not explain exactly how data-nosnippet applies to AI-generated answers.
Clearer guidance would help website owners understand how these controls interact and avoid making decisions based on assumptions.
A Common Way to Set AI Preferences
The Internet Engineering Task Force (IETF) has an AI Preferences working group drafting a common vocabulary for how content may be used.
If such an approach is adopted, it could provide a more consistent way to communicate AI preferences across different services, alongside Google’s own controls.
More Attention on Low-Value Content
Google continues to emphasise its policies on scaled content and user-generated content.
For large websites, keeping low-value pages out of Search is becoming an increasingly important part of managing website quality and indexing.
More Changes to Automated Search Data Collection
Google’s spam policies identify scraping search results for rank checking as machine-generated traffic. Google has also reportedly been returning altered results to suspected scrapers, as discussed in our poisoned SERPs analysis.
This makes third-party ranking data harder to rely on without additional checks. First-party tools such as Search Console’s URL Inspection are therefore increasingly important when you need to establish how Google sees a page.
About Szymaniak Digital
Szymaniak Digital is a UK enterprise SEO and AI search consultancy. We help businesses manage indexing across large websites, choose the right controls for each template, and agree AI visibility policies with marketing and legal teams.
We also help build release checks that catch unexpected indexing rules before they remove important pages from Google Search.
If you’re unsure what Google can index, display or use from your website, we can audit your templates, file types and settings and create a central register your team can use to manage them.
Get in touch with Szymaniak Digital to discuss your website’s indexing and AI search controls.
