Noindex is a meta robots tag or HTTP header directive that is used to tell search engines like Google, Bing, and others that a particular page should not be indexed and therefore not displayed in search results. By using noindex, you can effectively control which pages on your website should appear in search engine results and which should be excluded. However, these will still be displayed on your website.
What does Noindex mean and how does it work?
When a search engine crawls a page with a noindex tag in the code, it will still crawl the page and follow its links, but it will not include that page in its index. This is useful for pages that are accessible to users but not necessarily relevant to search engines, or that you don’t want to rank in search results for other reasons.
Noindex can be added to a page using meta tags in the HTML document or via HTTP headers.
Meta tag implementation
<meta name=”robots” content=”noindex”>
HTTP header implementation
X-Robots-Tag: noindex
When should you use Noindex?
There are several situations where it can be beneficial to use noindex on a page. Here are some of the most common:
Pages with low SEO value
Pages that do not contribute to your SEO strategy or are not important for users to find via search engines can be marked with noindex. Examples include:
Thank-you pages: After a conversion, such as a completed form or a completed purchase, the user is often directed to a thank-you page. These pages typically have low SEO value and there is no reason for them to appear in search results (they may also contain sensitive personal information).
Login pages: Pages where users must log in to access content should not appear in search results, as they do not provide value to users in search engines (these may also contain sensitive personal information).
Internal search pages: Many websites have an internal search function. Results pages from these searches usually do not have any unique or SEO-relevant information and should therefore avoid being indexed.
Duplicate content
Duplicate content can hurt your SEO strategy, as search engines can have difficulty determining which version of a page to show in search results. In some cases, it may be a good idea to use noindex to prevent certain pages from being indexed when they have duplicate content.
It may make sense to have pages with the same content for users.
Examples may include:
Filtering and sorting pages: Web stores that have pages based on different filters and sorting options can create many different URLs with largely the same content. By using noindex on these pages, you can prevent search engines from indexing them as separate pages.
Printer-friendly versions of pages: Some websites have printer-friendly versions of their pages that contain essentially the same content as the main page. These pages should be marked with noindex to avoid indexing issues and duplicate content.
Private or pages with sensitive information
Some pages contain information that should not be accessible via search engines, either because it is sensitive information about users or because it is only relevant to a limited audience. Noindex is an effective tool to keep such pages out of Google’s index.
Examples could be:
Internal documentation or FAQ pages that are only relevant to a specific user group.
Customer service portals or intranets where access is limited to authorized users.
Test and development sites
Pages used to test new designs or content should not appear in search results. These pages can also be marked with noindex until they are ready to go live. This ensures you avoid duplicate content and that users don’t see the new design until it is ready and live.
The relationship between Noindex and Nofollow
In addition to the noindex tag, there is also nofollow, which is used to prevent search engines from following links on a page. When you use noindex alone, search engines will still crawl and follow links on the page, but they will not index the page itself. If you want to prevent both indexing and crawling of links on the page, you can combine noindex and nofollow as follows:
<meta name=”robots” content=”noindex, nofollow”>
This combination prevents both indexing of the page and the crawler from following any links on the page.
How do you check if a page is Noindex?
To verify if a page has noindex, you can use several methods:
Google Search Console
Google Search Console allows you to see which pages Google has indexed, and you can also use the “Inspect URL” tool to check if a page is marked as noindex. This can provide valuable insight into how Google crawls and processes your pages.
Check the source code
To check the noindex status of a specific page, you can look in the HTML code for the meta tag. Go to the page, right-click and select “View Page Source” (or similar depending on your browser), and look for the following tag:
<meta name=”robots” content=”noindex”>
Using SEO tools
There are a number of SEO tools, such as Screaming Frog SEO Spider and Ahrefs, that can crawl your website and show you which pages have the noindex tag. These tools can also help identify unintentional use of noindex.
Noindex is an important SEO tool that allows you to control which pages Google and other search engines index. By using noindex correctly, you can prevent irrelevant or harmful pages from appearing in search results, while ensuring that the pages that should rank get the attention Google needs.
At Click in, we offer SEO advice on technical SEO, including how to best implement noindex to optimize your website’s visibility in search engines.
