Tools
Guides

Robots Meta Generator

Dev

Assemble the <meta name="robots"> tag and X-Robots-Tag header from index/noindex/follow/noarchive and other directives.

100% client-side No backend
Options
Meta tag
X-Robots-Tag header
On this page

What is a robots meta directive?#

A robots meta directive tells search engine crawlers what they may do with a page: index it or not, follow its links or not, keep a cached copy, show a snippet in results, and so on. It is the page-level lever you pull when a URL must stay public (a staging page, a thank-you page, a legal PDF) but should not appear in Google’s results, or when you want to shape how a result snippet looks.

There are two places a directive can live. The robots meta tag sits in the page’s HTML <head> and applies to that one HTML document. The X-Robots-Tag HTTP header travels in the response itself and applies to anything a server returns — a PDF, an image, a video, a JSON file — including resources that have no <head> to put a tag in. Search engines honour both; this tool generates both forms from the same set of toggles so you can copy whichever one your stack needs.

The single most important rule to remember: the absence of a directive means “no restriction.” A page with no robots tag at all is treated as index, follow — fully crawlable. The directives this tool emits are all opt-in restrictions on top of that default. Google’s reference defines the standard set, and that is exactly what the toggles here correspond to.

How to use it#

  1. Tick the Options checkboxes on the left. The first two are special — they default to checked:
    • index / follow — uncheck to emit noindex / nofollow.
    • noarchive — prevents Google from showing a “Cached” link.
    • nosnippet — suppresses the text snippet under the result.
    • noimageindex — stops this page’s images from being indexed as referring pages.
    • notranslate — declines the “Translate this page” offer in results.
    • noodp — legacy; ignores the obsolete DMOZ/ODP title.
    • nositelinkssearchbox — suppresses the sitelinks search box.
  2. Optionally pick a date and time under unavailable_after — the page will be dropped from search results after that moment. Useful for expiring a promotion or a time-limited document.
  3. Read the two outputs on the right:
    • Meta tag — the full <meta name="robots" content="..."> element, ready to paste into a <head>.
    • X-Robots-Tag — the HTTP header form, ready to add to your server or CDN response config.
  4. Click Copy next to either output to grab it.

Key features#

  • Both forms from one source. Meta tag and X-Robots-Tag header are generated together, so HTML pages and non-HTML resources (PDFs, images) get the same directive.
  • Default-aware. index and follow default to checked, matching the real web default of “no restriction unless stated”.
  • Full standard directive set. Covers every directive Google documents — noindex, nofollow, noarchive, nosnippet, noimageindex, notranslate, noodp, nositelinkssearchbox, and unavailable_after.
  • unavailable_after support. The only directive that takes a value; it is assembled as unavailable_after: <date> and passed through verbatim so you control the exact format.
  • 100% client-side. No config to upload, no server to round-trip — generate and copy, then apply the result in your CMS, template, or nginx config yourself.

Worked example#

Leave the defaults (only index and follow checked) and both outputs read:

<meta name="robots" content="index, follow">
X-Robots-Tag: index, follow

That is the explicit form of “no restriction” — useful when you want to be deliberate about a page being fully crawlable.

Now uncheck index (to keep a thank-you page out of the index while still passing link equity through its links) and the outputs become:

<meta name="robots" content="noindex, follow">
X-Robots-Tag: noindex, follow

A third case: a landing page for a seasonal campaign that must disappear after it ends. Keep index and follow checked, set unavailable_after to the end of the campaign, and the tag becomes:

<meta name="robots" content="index, follow, unavailable_after: 2025-12-31T00:00">

Google will stop showing the page in results after that timestamp. The date value is emitted exactly as you selected it, so confirm it matches a format your target search engine accepts.

FAQ#

What is the difference between noindex in a meta tag and disallowing in robots.txt?#

They work at different layers and must not be mixed carelessly. noindex is a directive the crawler reads after fetching the page. If you block the URL in robots.txt, the crawler never fetches it, never sees the noindex, and may still index the URL based on links pointing at it. To keep a page out of the index reliably, allow crawling but add noindex — do not hide the page from the crawler entirely.

When should I use the X-Robots-Tag header instead of the meta tag?#

Whenever the resource is not HTML — a PDF, an image, a video, a spreadsheet — there is no <head> element to put a meta tag in, so the header is the only option. It is also the right choice for applying one rule to many files at once via your server or CDN config, rather than editing each document.

Yes. noindex controls whether the page appears in results; follow controls whether the crawler follows the links on the page. noindex, follow keeps the page out of the index but still lets link equity flow through the page’s outgoing links. This is the standard recipe for thank-you pages, internal search results, and faceted-filter pages.

Why does the tool emit index, follow at all when that is the default?#

Because being explicit can be useful: it documents intent in the page source, it can override a more restrictive inherited rule, and some teams treat the presence of the tag as a signal that the page was reviewed. If you prefer to rely on the implicit default, simply leave the tag out — omitting it entirely is equivalent to index, follow.