Meta Robots tag essentials
- The concept: An invisible line of code that tells search engines what they can do with a page
- Key commands : noindex (do not show on Google) and nofollow (do not follow links)
- Utility: Hide pages with no SEO value to concentrate power on your strategic pages
- The risk: A configuration error can make your site invisible
Visit SEOit's not just about trying to be number 1 everywhere. It's also about telling Google: "This page is none of your business". The Meta Robots is your remote control: it allows you to direct the robots that crawl your site to focus only on what brings in business.
For an executive or marketing manager, mastering this tool is a matter of cleanliness and security. It's the best way to prevent draft pages, internal search results or confidential documents from ending up freely available on the web.
How does this "remote control" work?
The Meta Robots tag is a small line of code in the HTML header of every page. It changes nothing for your human visitors, but it changes everything for Google, Bing or Yahoo robots.
Index / Noindex
- Control the right of entry
- "noindex" = page not displayed in Google
- Ideal for : T&Cs, acknowledgements, login
Follow / Nofollow
- Control link tracking
- "nofollow" = do not transmit authority
- Ideal for: invalid external links
3 concrete cases where you need to take action
- Test or pre-production pages : Nothing is more annoying than seeing a test version appear in front of your official site. The "noindex" is mandatory here
- Visit duplicate content : If you have two pages that are almost identical for technical reasons, set one of them to "noindex" to avoid penalties
- Sensitive member areas or PDF : If you don't want your internal documents to be indexed, the robots tag is your first line of defence
Please note: A simple error in your CMS can check the "noindex" box for your entire site. It's an industrial accident: your traffic drops to zero in a matter of days. Keep a close eye on your tags.