How to solve the problem of sitemap contains URLs which are blocked by robot.txt in google search console in Prestashop 1.7.x after upgrade from previous version?

How to solve the problem of sitemap contains URLs which are blocked by robot.txt in google search console in Prestashop 1.7.x after upgrade from previous version?

Before we go solution steps, we need to understand what the Robots.txt file is?

Robots.txt is a pure text file that asks search engine what URLs or contents are supposed to be crawled. While publishing a new page or posting some content to your website, search engine bots can crawl this material to index it to the search results. At the same time, If you do have certain parts of your website that you don’t want to be indexed, you can tell search bots to skip them so that they don’t show on the search results page.

Let’s go back to our solution for Prestshop 1.7.x. when you see the errors in your Google search console like “URL blocked by robots.txt” and “URL is not on Google: indexing errors”. Attached screenshot is a reference.

This kind of error will happen sometimes when you do the Prestashop version upgrade, migration or any other. Please don’t worry and just follow the below steps.

Part I
1/. type www.yourdomain.com/robots.txt to verify whether you have this file or not?
2/. Make sure the following lines are available in your Robots.txt files
Allow: /modules/.css
Allow: /modules/.js
Allow: /modules/.png
Allow: /modules/.jpg
Allow: /js/jquery/*

3/. You can use the below link to check your Robots.
https://www.google.com/webmasters/tools/robots-testing-tool?pli=1

4/. If your domain is not appear in your property selection, you can use the following url to add it
https://www.google.com/webmasters/tools/robots-testing-tool?hl=en&siteUrl=https://www.yourdomain.com
5/. You also need to check if you can find any sitemap files like below in your backend. It is better to remove all the sitemap files first. We will re-generate it.
1_en_0_sitemap.xml
1_index_sitemap.xml
1_tw_0_sitemap.xml

Part II
1/. The following we will re-generate Robots.txt.
2/. Log into your Prestashop admin portal. Go to “Configure” section, under submenu “Shop Parameters” and “Traffic & SEO”, scroll down to bottom and click “Generate robots.txt file”


3/. https://www.google.com/webmasters/tools/robots-testing-tool?pli=1 select your domain property
4/. In robots.txt Tester page, scroll down to the bottom and put “robots.txt” in the Enter a URL to test if it is blocked field. Then click “Test” red button to release the updated version.


5/. You will see “Latest version seen on date and time OK (200) xxxx Bytes”

Part III
1/. Regaring how to generate a sitemap, you may refer to the below url.
https://www.genkiware.com/2020/02/11/how-to-generate-a-google-sitemap-in-prestashop-1-7-x/

** Attention: Make a full backup prior to the change

There are no reviews yet.

Leave a Reply

Your email address will not be published. Required fields are marked *



Start typing and press Enter to search