Class 19 - XML, TXT, RSS, ATOM Sitemap | FREE Advanced SEO Course With Notes
Watch the Full Video: XML, TXT, RSS, ATOM Sitemap
A sitemap is the only thing that touches every aspect of our SEO journey. It helps in the discovery of our website, it helps in crawling, it helps in indexing, and it also helps in ranking. So, it is very important to use the sitemap properly. It is a very important file.
Types of Sitemaps
HTML Sitemap: An HTML sitemap is created on a website where all the URLs of the website are listed. This HTML sitemap is not for Googlebot or SEO this sitemap is for users. By looking at the HTML sitemap, users can understand which pages and categories are on the website, etc. But HTML sitemaps were created earlier now this sitemap is not created on websites.
XML Sitemap: Most used format in SEO. A sitemap that is in XML format. XML sitemap is the most popular, and Google also gives the highest preference to XML sitemaps.
Google also accepts TXT format sitemaps. You can add the URLs of pages in a TXT file and submit it. Also, Google accepts RSS and ATOM types of sitemaps.
Some Rules of Sitemap
- You can include a maximum of 50,000 URLs in any sitemap. If there are more than 50,000 URLs, you should create another sitemap.
- The size of a sitemap file should not be more than 50 MB.
- Your sitemap should not be disallowed in the robots.txt file.
Unlike the robots.txt file, the sitemap is not located in the website’s root folder. You can also save the sitemap in subfolders. Any folder, like “mitromate.com/sitemaps/sitemap.xml”. You have to submit this URL in Google Search Console and the robots.txt file so that Google can confidently process that sitemap.
Sitemap Index
A sitemap index file is the parent file of all sitemap files, in which all the sitemaps are included, not the pages. If you want to see it, look at Google’s sitemap, and you will understand. I am giving you the link here: “https://www.google.com/sitemap.xml“.
If you create a sitemap index file, then in Google Search Console you only have to submit this sitemap index file. The rest of the sitemaps will be added automatically.
The same rule applies to the sitemap index file. You can add only 50,000 sitemaps in it, not more than that, and the file size should not be more than 50 MB.
As you know, one sitemap has a limit of 50,000 URLs, and the sitemap file should be 50 MB. Now, suppose your website has 60,000 pages, then what will you do? We will create a second sitemap.
“mitromate.com/sitemaps/sitemap.com (1 – 50,000 URLs)”
“mitromate.com/sitemaps/sitemap2.com (50,001 to 60,000 URLs)”
How to Create a Sitemap?
The method of creating a sitemap is very simple:
Plugins (CMS): You can create a sitemap by installing a sitemap plugin. If Rank Math or Yoast is installed, the sitemap is created automatically.
Direct Method: In this method, we can ask the developer to create a sitemap. You just have to explain the sitemap structure. There can be one problem here: whenever we add a new URL, we will have to add that URL to the sitemap manually.
Tools: If your website is small, you can use free online tools to create a sitemap. On https://www.xml-sitemaps.com/, whatever file you get from there, you have to upload it to your hosting. Then, whatever URL you get, you have to submit it in Google Search Console.
Sitemap Elements
<urlset>: This is the first element of the sitemap. It is the element inside which the entire sitemap is placed.
<url>: Inside <urlset>, there is a <url> tag for each individual URL, in which different URLs are added individually. Inside the <url> tag, there are 4 other tags: <loc>, <lastmod>, <changefreq>, and <priority>.
<loc>: Means the exact URL that we are adding to the sitemap.
<lastmod>: The <lastmod> tag tells the search engine through the sitemap on which date and at what time this URL was last modified.
<changefreq>: It means we tell the search engine how frequently this page is changed. Google does not use the change frequency tag because Google decides this by itself.
<priority>: It means how important this page is for my website. Google does not consider the priority tag because Google decides by itself how much priority to give to a page.
Sitemap Appears to be an HTML Page
This is an issue that is sometimes seen in Google Search Console when we add a sitemap. This happens for two or three reasons.
Your sitemap URL has been cached by a caching plugin. The solution is to go to your caching plugin and exclude your sitemap URL from caching.
If the sitemap structure is not correct, this issue can also occur. The solution is to use a sitemap validator and enter your sitemap URL. You will know if there are any issues with your sitemap structure. Another way is, if your website is on WordPress, go to Settings, click on Permalinks, and at the bottom there will be a Save Changes button. Just click it. What this will do is flush all the URLs, and your sitemap will be generated again.
Couldn’t Fetch Sitemap Error in Google Search Console
Now, how do you solve the “Couldn’t Fetch” issue? First of all, if you have recently added the sitemap and this error appears, then wait for a while because sometimes it gets solved in 15 to 20 minutes.
If this issue still appears after waiting, then what you have to do is inspect your sitemap URL using the Google Search Console URL Inspection tool. There, it will show as not indexed. What you have to do is click on the “Test Live URL” button and then check whether “Crawl Allowed?” is “Yes” or “No”. It should be “Yes”. Also, “Page fetch” should be “Successful”. If crawl is allowed and the page is being fetched, it means Googlebot is reaching your page. There is no problem, it will be processed.
Your sitemap URL should not be blocked in the robots.txt file. If it is, your sitemap will never be indexed successfully.
Your website should not have a manual action. If your website has a manual action or penalty, Google will not crawl your sitemap.
Your sitemap URL should be correct, otherwise it will not be crawled. Sometimes something may be missing in the URL, so make sure that your sitemap URL is correct.
If your website is new and there is no content on your website, this issue can also occur. So, wait for some time. When Google feels that your website has enough content, it will crawl your sitemap.
Make sure that your sitemap URL is not redirected. If it is, this error can also occur. You can also check your sitemap using the https://httpstatus.io/ tool. There, you will know whether your sitemap status code is 2