Is Your Website’s XML Sitemap Holding Your Site Back?
“Add a sitemap, and Google will index everything!”
If only it were that simple. XML sitemaps are some of the most misunderstood tools in website optimization.
Yet their proper implementation can dramatically influence how search engines perceive and crawl your site.
What Is an XML Sitemap?
An XML sitemap acts as your website’s directory for search engines.
While regular sitemaps help human visitors navigate your website, XML sitemaps help search engines understand your website’s structure and content.
Here’s what a sitemap looks like:
When you add a new product page, publish a blog post, or update your service offerings, you need Google (and other engines) to find and index that content as quickly as possible.
Index
An index is a computer-generated list of every page on a website that can be accessed by a search engine. It is created by web crawlers and used by search engines to find pages when users type in queries.
Read More
What Does a Sitemap Include?
Your XML sitemap tells Google four important things about every page.
1. Location (loc)
This is the URL of the page, and this needs to be the full URL.
<loc>https://www.dreamhost.com/products/blue-widget</loc>
2. Last Modified Date (lastmod)
A timestamp showing when content was updated.
Microsoft’s Bing team emphasizes that including the lastmod tag is “crucial” for effective crawling.
Google specifically looks for meaningful updates.
According to their documentation, “Google uses the <lastmod> value if it’s consistently and verifiably (for example, by comparing to the last modification of the page) accurate.”
<lastmod>2024-11-30T14:30:00+00:00</lastmod>
Google also explains how to view lastmod dates for page updates:
“The value should reflect the date and time of the last significant update to the page. For example, an update to the main content, the structured data, or links on the page is generally considered significant; however, an update to the copyright date is not.”
3. Change Frequency (changefreq)
How often the content typically updates. Here are some example values:
- For your homepage: “daily” or “always”
- For product pages: “weekly”
- For blog posts: “monthly”
- For about pages: “yearly”
- Misc pages: “never”
<changefreq>weekly</changefreq>
4. Priority (priority)
The relative importance compared to other pages. This value can range from 0 to 1:
- Homepage: 1.0
- Main category pages: 0.8
- Product pages: 0.6
- Blog posts: 0.4
- Archive pages: 0.2
<priority>0.8</priority>
Does Your Website Need an XML Sitemap?
We’d highly recommend you create a sitemap for your website. While most people would suggest having a sitemap only if you have:
- “A really large website”
- “Websites with a large archive”
- “A new website with just a few links pointing to it”
- “A website which uses rich media content”
Google maintains that with proper internal linking, their crawlers should find your content naturally.
The reality? Most websites don’t achieve perfect internal linking structures. Every modern website benefits from having an XML sitemap.
Here’s why:
The internet grows more complex each day, making proper site indexing increasingly challenging. You should try to provide search engines with every possible detail to help them find your content.
XML sitemaps don’t just help with discovery; they make the entire crawling process more efficient and help make optimum use of the crawl budget.
What Pages Should You Add to the Sitemap?
Your website contains two distinct types of pages.
1. Search landing pages
Pages users should find through search:
- Product pages
- Category listings
- Blog posts
- Service descriptions
- Key information pages
2. Utility pages
Pages that serve a function but shouldn’t appear in search:
- Login screens
- Order confirmation pages
- Password reset forms
- Thank you pages
- Admin sections
Your XML sitemap should only include the search landing pages. Including utility pages dilutes your site’s perceived quality and wastes valuable crawl resources of the search engine.
How To Create XML Sitemaps?
You have several options for creating XML sitemaps, ranging from manual creation to automated solutions.
Let’s start with the simplest approaches and move to more advanced methods.
Manually Creating a Sitemap
For small static websites, you can manually create your XML sitemap using any text editor.
Create a new file called “sitemap.xml” and use this basic structure:
<?xml version=”1.0″ encoding=”UTF-8″?>
<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
<url>
<loc>https://www.yourwebsite.com</loc>
<lastmod>2024-01-01</lastmod>
</url>
</urlset>
Add a new <url> block for each page you want to include. While this method works for small sites, it becomes impractical as your website grows.
Automated Solutions for WordPress Sites
If you run a WordPress website, you already have several powerful SEO tools:
- All in One SEO: Offers comprehensive sitemap features, including custom post type support and automatic priority calculation based on content age and type. Access your sitemap at “yourdomain.com/sitemap.xml.”
- Yoast SEO: The most popular choice for WordPress sitemap generation. After installation, your sitemap is generated automatically at “yourdomain.com/sitemap_index.xml.” You can control which content types appear in your sitemap through the plugin’s settings.
- Rank Math: Offers similar functionality to Yoast but includes additional features like Google News sitemaps and video sitemaps. Your sitemap will be available at “yourdomain.com/sitemap.xml.”
- WordPress Core: Since version 5.5, WordPress includes basic sitemap functionality. However, most SEO professionals prefer dedicated plugins for their additional control and features.
Solutions for Non-WordPress Websites
If you used another platform to build your website, they likely have a built-in solution for creating these sitemaps.
If you don’t use a platform and have built a fully custom website, you’ll need to rely on an external sitemap generator and upload the auto-generated sitemap to your web server at regular intervals.
Here are a few good sitemap generators:
- XML-Sitemaps.com: Upload your site structure or let it crawl your website to generate a sitemap.
- Screaming Frog: Crawls your website and creates a sitemap based on your specifications.
- Sitebulb Website Crawler: Streamlines XML sitemap creation with powerful insights to optimize site structure.
How To Submit Your Sitemap to Google
Generating your sitemap marks only the beginning of the process. You need to tell Google where to find it.
While Google might eventually discover your sitemap through your robots.txt file, direct submission through Google Search Console speeds up the process significantly.
Before You Submit
First, ensure your sitemap actually works. A broken sitemap wastes Google’s time and yours. Visit “yourdomain.com/sitemap.xml” in your browser. You should see a properly formatted XML file, not an error page.
Pro Tip: If you see garbled text, don’t worry — that’s normal. XML files often look messy in browsers. The important part is that you can see your URLs listed.
Submitting Through Google Search Console
- Log into Google Search Console.
- Select your website property.
- Navigate to the Sitemaps section from the left sidebar.
- Enter your sitemap URL in the “Add a new sitemap” field and click SUBMIT.
Depending on if you have a domain property or URL prefix, you’ll either need to enter the full URL or just the part after your domain.
For instance, in the above image, you see that we entered the full URL since it’s a domain property.
Here’s what it looks like for a URL prefix property — here, you only need to enter the part after the domain name:
You’ll typically store the XML sitemap as “https://yourdomain.com/sitemap.xml” or “sitemap_index.xml.”
Google will immediately check your sitemap for basic errors. You’ll see one of these status indicators:
- Success: Google can read your sitemap correctly.
- Couldn’t fetch: Google couldn’t access your sitemap.
- Couldn’t read: Google found your sitemap but couldn’t process it.
Understanding the Sitemap Report
Once submitted, Google Search Console shows you:
- How many URLs you’ve submitted.
- How many URLs Google has actually indexed.
- The last time Google read your sitemap.
Important: Don’t panic if the number of indexed URLs differs from submitted URLs! Google doesn’t index pages just because they’re in your sitemap — remember, it’s a suggestion, not a command.
Multiple Sitemaps
For larger websites, you might have several sitemaps:
- Product sitemap
- Blog post sitemap
- Category sitemap
- Image sitemap
Submit each one separately to help Google understand your site structure better and to make tracking indexation issues easier.
Submitting Your Sitemap to Bing
While Google dominates the search landscape, smart SEO professionals know the value of diversifying their search engine presence. Bing captures a significant portion of search traffic, especially in certain demographics and regions. Plus, their webmaster tools often provide unique insights you won’t find elsewhere.
You’ll notice that when you click on “Add new site”, it gives you two choices – “Import your sites from GSC,” or “Add your site manually.”
Importing Websites From Google Search Console
Already verified your site in Google Search Console? You’re in luck! Bing offers a streamlined import process that can save you valuable setup time.
Manual Site and Sitemap Submission
For those starting fresh or preferring manual control, Bing’s direct submission process proves straightforward:
- Navigate to Bing Webmaster Tools.
- Choose Add your site manually.
- Enter your site’s URL.
- Verify ownership through your preferred method:
- Upload an XML file.
- Add a meta tag.
- Add a CNAME record.
Head to Sitemaps from the left sidebar and then click Submit sitemap.
Finally, add the full sitemap XML and click Submit.
Common Submission Errors
For Bing Webmaster tools, you just see the word “Error” or “Success,” and clicking on the row will give you more details about the error.
For Google Search Console, however, you’ll see errors right there.
If you get “Couldn’t fetch,” check whether:
- Your sitemap URL is correct.
- Your website is accessible.
- Your robots.txt file isn’t blocking the sitemap.
If you get “Couldn’t read,” verify that:
- Your XML syntax is valid.
- You’re using the correct XML schema.
- The file isn’t compressed or corrupted.
After Submission
Monitor your sitemap status regularly. Pay attention to:
- Sharp drops in indexed pages.
- Growing gaps between submitted and indexed URLs.
- Warning messages in the sitemap report.
Remember: Submitting your sitemap to Google represents an ongoing process, not a one-time task. Keep monitoring and updating as your website grows and changes.
Common Myths About XML Sitemaps
Let’s clear up some persistent misconceptions about XML sitemaps and their role in search engine optimization.
Myth 1: Sitemaps Ensure Indexing on Google
Many website owners believe submitting a sitemap means automatic indexing. But adding pages to your sitemap doesn’t guarantee Google will include them in search results.
Google’s algorithm decides what to index based on many factors, including page quality, uniqueness, and overall value to users. Your sitemap only serves as a gentle suggestion to Google saying, “Hey, these pages are worth your attention.”
Much like adding a page to your main navigation menu, it’s one of many signals Google uses to understand your site’s structure and content hierarchy.
Myth 2: Every Page Should Be in Your Sitemap
Often, website owners add their entire site structure into their sitemap, thinking more is better.
Your sitemap should only include pages you genuinely want users to find through search. Try to maintain its size under 50,000 URLs and keep file sizes below 50MB uncompressed.
UTF-8 encoding also helps search engines read your sitemap correctly, while proper XML formatting prevents parsing errors that could derail your efforts.
The thing is, if you start including login pages, thank you pages, or duplicate content; it indicates to Google that you don’t understand what content actually matters to users.
Think of your sitemap like a restaurant menu. You wouldn’t list the kitchen, storage room, or staff bathroom — you only show what customers could order.
Myth 3: Update Frequency Doesn’t Matter
Sitemaps are living documents.
Even if manually updating them seems like a chore, you need to use some form of automation to auto-update the sitemap with any newly published pages that matter to you.
Outdated sitemaps containing deleted pages or missing new content can actually harm your site’s crawling efficiency. Google’s crawlers waste valuable time checking non-existent pages while potentially missing your fresh content.
Your sitemap should reflect your website’s current state, just like a map needs to show current roads and landmarks to be useful.
Myth 4: All XML Tags Are Equally Important
Many spend time perfecting their changefreq and priority values, thinking they significantly influence Google’s behavior.
The URL location (loc) and last modified date (lastmod) are the only two required elements of your XML sitemap. The other tags provide context but don’t directly impact crawling or indexing decisions.
So, focus your energy on maintaining accurate URLs and last-modified dates, and you can completely skip worrying about priority and change frequency.
Final Thoughts
Your XML sitemap tells a story about your website’s organization and priorities. So, make it worth the search engine’s time to read and process it.
And here’s one thing to take away: quality beats quantity here. A focused sitemap of your best pages outperforms a cluttered directory of everything you own.
Want to nail the technical foundation of your website?
Dreamhost’s managed WordPress hosting handles the complexities while you focus on what matters — creating content your audience loves.
This page contains affiliate links. This means we may earn a commission if you purchase services through our link without any extra cost to you.
Did you enjoy this article?
