How to Deindex a Page From Google (and the robots.txt Trap That Keeps It Indexed)

To deindex a page from Google you add a noindex rule and you keep the page crawlable, because a page blocked in robots.txt never gets read, so Googlebot never sees the rule and the URL can sit in the index anyway.

Below are the methods, the scenario each one actually fits, the trap that quietly undoes most attempts, and the verification loop nobody bothers to show you.

What deindexing a page from Google actually means

Deindexing means removing a page from Google’s index so it can no longer be served as a search result, which is a different thing from blocking the crawl. Crawling is Googlebot fetching the page; indexing is Google storing it and making it eligible to show. You can be crawled and not indexed, and you can be blocked from crawling and still indexed.

That last one catches almost everyone, and it is the whole reason this article exists.

Before you deindex: should the page be consolidated instead?

If the page has any links, impressions, or history, consolidating it into a stronger page with a 301 redirect or a canonical tag is usually the better move, because a redirect keeps the equity the URL earned while deindexing simply throws it away.

Not one page in the search results for this topic asks you to make that call first, and it is the call that matters most. Deindexing is permanent housekeeping, so run through this quickly before you reach for a tag:

• It is a near-duplicate of a better page. Merge it and 301 the old URL, or set a canonical pointing at the version you want kept. You keep the links; see our guide to duplicate posts and pages in WordPress.
• It is thin but the topic still matters. Expand it rather than kill it. A deindexed page earns nothing forever.
• It is genuinely junk or private. Now deindexing is the right answer, and the rest of this article is for you.
• It was never meant to be indexed anyway. Some pages should simply stay out, and Google’s own indexing guidance is comfortable with that. We covered the healthy version of this in crawled, currently not indexed.

The methods to deindex a page from Google

There are five methods, and the first one covers the large majority of cases. Google’s block indexing documentation is unusually plain about which is which.

1. The noindex meta tag, for normal HTML pages

Put this in the page’s En-tête>. Google’s doc says that when Googlebot crawls the page and extracts the tag, “Google will drop that page entirely from Google Search results, regardless of whether other sites link to it.”

<meta name="robots" content="noindex">

Utilisation googlebot au lieu de robots if you want to block only Google and leave other engines alone. Most of the time you do not.

2. The X-Robots-Tag header, for PDFs and other non-HTML files

A PDF has no En-tête>, so there is nowhere to put a meta tag. Return the rule as an HTTP response header instead, which Google’s doc explicitly recommends “for non-HTML resources, such as PDFs, video files, and image files.”

X-Robots-Tag: noindex

You set this at the server level, in your Apache config, your nginx config, or via your host’s headers panel. Both methods have the same effect, and Google says to pick whichever suits the content type.

3. Delete the content, returning a 404 or a 410

If the content should not exist at all, removing it is the most complete option, and Google’s page removal doc calls it “the most secure way to prevent your information from appearing in other search engines that might not respect the noindex tag.”

It also stops people reaching the page directly, which a noindex does not.

4. Password-protect the page

Putting the page behind authentication keeps the right people in and every crawler out. This is the right answer for staging sites, client portals, and anything genuinely private, because it does not rely on a crawler choosing to obey a rule.

5. The Search Console Removals tool, for urgent cases only

The Removals tool hides a URL from Google’s results within about a day, which is why it exists, but it is temporary. Google’s doc is direct about the limit: “Requests made in the Removals tool last for about 6 months.”

So treat it as an emergency brake, never as the fix. Fire it when something sensitive is live in the results right now, then immediately put a permanent method behind it, or the URL comes back in six months. Our older guide to removing URLs from Google search results covers this tool in more depth.

One detail from Google’s doc that almost nobody repeats: protect every variation of the URL, because different URLs can point to the same page. Google’s own examples are example.com/puppies, example.com/PUPPIESet example.com/petchooser?pet=puppies. Removing one and leaving the others is a common way to think you are done when you are not.

The robots.txt trap that keeps a page indexed

Adding noindex to a page and also blocking that page in robots.txt does not double your protection, it cancels it, because Googlebot has to crawl the page to see the tag and robots.txt is what stops it crawling. The tag never gets read and the URL can stay in the index.

Google puts this in a box marked “Important” in its own documentation: “For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file… the crawler will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.”

And the robots.txt introduction opens by telling you the file was never for this: “it is not a mechanism for keeping a web page out of Google. To keep a web page out of Google, block indexing with noindex or password-protect the page.”

Here is what you actually get when you block a page in robots.txt and it has links pointing at it. Google can index the URL without ever reading the page, so the result shows up with the URL, possibly the anchor text people used to link to it, and no description at all. That ugly, descriptionless result is the classic symptom.

The fix is simple once you see it: let Googlebot crawl the page, keep the noindex on it, and wait for the recrawl. Add the robots.txt block later, if you even still want it, once the page has actually dropped out.

Which deindexing method fits your situation

How to deindex a page in WordPress

In WordPress you do not touch the theme files, you flip a per-page switch in whichever SEO plugin you run, and it writes the noindex meta tag into the head for you.

• Rang Math: open the post or page, find the plugin’s sidebar panel, go to the Advanced tab, and set the Robots Meta to No Index. It is one checkbox, and on most builds it is the fastest route.
• Yoast SEO: open the post, find the Yoast box, go to the Advanced tab, and set “Allow search engines to show this page in search results?” to No. Same result, different wording.
• Whole sections at once: both plugins let you noindex entire archive types (tag archives, author archives, paginated pages), which is usually where the real index bloat is hiding.

After you flip the switch, view the page source and confirm the tag is actually there. Plugins conflict, caching layers serve stale HTML, and a switch you flipped in the editor is not the same thing as a tag Googlebot receives.

How long does it take Google to deindex a page?

It depends entirely on how often Google crawls that URL, and it can take months. That is not me hedging, it is Google’s own answer: “Depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page.”

You will see “a few days” quoted all over this topic. On a busy, well-linked page that is often true. On the forgotten tag archive you are trying to clean up, it is not, and that page is exactly the kind Googlebot is in no hurry to revisit.

So do the one thing that actually moves it: add the tag, then request a recrawl through the URL Inspection tool for the URLs you care about. For a handful of important pages that is worth the clicks. For 500 tag archives, add the rule and be patient.

How to check the page was actually deindexed

Two screens in Search Console close the loop, and between them they tell you whether Googlebot saw your rule and whether it acted on it.

URL Inspection tells you what Googlebot received. Inspect the URL and open the crawled HTML view, which shows the page exactly as Googlebot fetched it. If your noindex is not in that HTML, it does not exist as far as Google is concerned, no matter what your browser shows. Google recommends this exact check for this exact purpose.

The Page Indexing report tells you it worked. Under “Why pages aren’t indexed” there is a bucket called “Excluded by ‘noindex’ tag”, and your deindexed URLs land there once Google has processed them. Here is that report on our own property.

Notice both rows in that screenshot. “Excluded by ‘noindex’ tag” means the system worked exactly as intended. “Blocked by robots.txt” means Google could not read those pages, so it cannot honor any indexing rule on them, and that bucket is where the trap shows up on a real site.

If your total indexed count moves and you are not sure why, we walk through reading that number properly in Search Console indexed pages decreased.

404 or 410: does the difference matter?

For permanent removal, Google’s own guidance treats a 404 and a 410 as interchangeable, and its removal doc simply tells you to remove the content without distinguishing between them.

Plenty of practitioners prefer 410, on the reasoning that “Gone” is a stronger signal than “Not Found” and should get the URL dropped faster. That is a sensible instinct and it is not what the documentation says.

Honestly, we have not measured it, so I am not going to pick a winner for you. If your server makes 410 easy, use it and lose nothing. If it does not, a 404 is fine and you should spend the effort somewhere that pays.

So what is the right way to deindex a page from Google?

Honestly, for almost every page it is the boring one: add noindex, leave the page crawlable, request a recrawl if the page matters, and check the Page Indexing report in a couple of weeks. The Removals tool is for genuine emergencies, and if you use it without a permanent method behind it you have bought yourself six months and a future surprise.

The mistake I see most is the one nobody warns about, and it is not choosing the wrong tag. It is reaching for robots.txt because it feels like the stronger, more decisive block, when it is the one thing that guarantees Google never reads your instruction.

And before any of that, ask whether the page deserves a redirect instead. Deindexing is easy to do and impossible to un-earn, so consolidating a page that still has something to give is usually the smarter call.

---

Share Copier le lien Copier au format Markdown

Want our posts to show up more often on Google?

One step & Google will surface this site in your Top Stories.

Add us as a preferred source on Google

Abdullah Nouman eCommerce SEO, Agentic Commerce SEO & AI Search Specialist

I publish the industry's most critical SEO and AI Search blog, cutting through the hype and explaining what actually holds up in plain English.

Venez travailler avec moi

Lectures complémentaires

Retour à tous les articles →

How to Improve INP on WordPress (Interaction to Next Paint) CrUX Vis vs PageSpeed Insights: Which Core Web Vitals Data to Trust How to Migrate a WordPress Site Without Breaking Images, Links, or SEO