X-Robots-Tag Header Generator

Generate X-Robots-Tag HTTP headers and matching robots meta tags for noindex, snippets, previews and crawler-specific rules.

← SEO Tools

Build an X-Robots-Tag HTTP header and matching robots meta tag from current indexing and preview directives.

SEOCrawlingBrowser processing

-1 lets Google choose; 0 means no text snippet.

· · · · ·

When to use X-Robots-Tag

Use the HTTP header when you need page-level indexing rules for non-HTML files such as PDFs or images, or when server configuration is a better fit than an HTML meta tag. Google states that rules used in a robots meta tag can also be sent with X-Robots-Tag.

Important

The URL must still be crawlable for Google to discover an X-Robots-Tag or robots meta rule. indexifembedded only has an effect with noindex. Google lists noarchive as historical and unused, so this generator does not add it.

Reference: Google Search Central robots meta and X-Robots-Tag specification.

Related tools