Find and Add Your Sitemap

RAMP will automatically crawl your websites to find pages to scan. However, sometimes you need to add pages yourself if the RAMP bot is blocked. One of the ways you can do this is by adding your XML Sitemap path.

In this article, we’ll cover:

What is an XML Sitemap?

How to Find Your Sitemap

How to Create a Sitemap (in case you don’t have one yet)

How to Add Your Sitemap to RAMP

Troubleshooting Sitemap Import Issues

Alternative Methods for Adding Pages

Complimentary Account Setup

What is an XML Sitemap?

An XML Sitemap is a file that lists your site’s URLs so that search engines can quickly and effectively crawl it. Sitemaps help search engines and other website crawlers, like Google and AccessibleWebBot, find and understand your website’s URLs, the content on each page, and the relationships between pages.

Alongside each URL, XML Sitemaps also list additional metadata including:

• When the page was last updated
• How often the page typically changes
• The priority of each page relative to other URLs on the site

Additionally, sitemaps provide information about specific content on your pages, like videos, images, and news articles. If your website is really large or has lots of rich media (video, images, news), having a sitemap is especially important.

When properly configured, your XML sitemap will stay updated with new pages as they’re added to the site.  It’s a dynamic inventory that changes with your website over time.

We recommend using a sitemap to import pages into Accessible Web RAMP for page monitoring. Adding your sitemap to RAMP is also the best way to make sure RAMP stays up-to-date when changes are made to your site and to ensure no pages of your site have been missed.

1. Manually Check the Most Common Sitemap Locations
2. Check Your /robots.txt File
3. Locate Wix, Squarespace, Shopify, and Blogger Sitemaps

Don’t have a sitemap? Jump to Creating A Sitemap

Video Tutorial

1. Manually Check the Most Common Sitemap Locations

The simplest way to find your sitemap is to manually check the most common sitemap locations. All you need to do is enter your URL in a browser and then try out a few different sitemap path variations at the end of it.

First, check the most common sitemap paths:

• /sitemap.xml
• /sitemap_index.xml
• /sitemap/

Example: https://yourdomain.com/sitemap.xml

Most popular CMS providers–including Wix, Squarespace, Shopify, and Blogger–automatically generate sitemaps or sitemap indexes for all websites built on their platforms. If your site is built on Wix, Squarespace, Shopify, or Blogger, your XML Sitemap or Sitemap Index should be located at /sitemap.xml.

Sitemap Index files are used to group more than one sitemap. Our website, for example, uses a /sitemap_index.xml file to organize our 8 different sitemaps. Searching https://accessibleweb.com/sitemap.xml in a browser redirects to our sitemap index at https://accessibleweb.com/sitemap_index.xml.

Accessible Web RAMP is able to ingest entire Sitemap Index files. All you need to do is enter your Sitemap Index’s URL path under Settings > Page Monitoring within RAMP. To scan only a subset of pages instead of the entire sitemap index, add one of your sub-sitemap paths (like /page-sitemap.xml) to RAMP instead of the entire sitemap index (/sitemap-index.xml).

2. Check Your /robots.txt File

Checking your robots.txt file is the next quick and easy method for locating your sitemap.

Sometimes, the robots.txt file also lists the sitemap URL. This can be especially useful when a website has a less common sitemap location.

To view your website’s robots.txt file, just add /robots.txt to your domain.

3. Locate Wix, Squarespace, Shopify, and Blogger Sitemaps

Many popular CMS automatically generate sitemaps. Wix, Shopify, Squarespace, and Blogger all generate a default XML Sitemap or XML Sitemap Index for websites built on their respective platforms.

If your website is built on Wix, Shopify, Squarespace, or Blogger, your sitemap path should be /sitemap.xml.

You will find your Sitemap or Sitemap Index located at www.yourdomain.com/sitemap.xml.

Read more about Wix Sitemaps from Wix Help Center…

Read more about Squarespace Sitemaps…

Read more about Blogger Sitemaps…

Learn how to customize your Shopify site’s robots.txt file…

Unable to locate your sitemap?

If you’re unable to locate a sitemap for your website using these first three methods, you might not have one yet. Luckily, creating a sitemap is no big deal.

How to Create a Sitemap

If your site doesn’t have a sitemap already, you can quickly and easily create one using one of the following methods.

1. Use Your CMS to Generate a Sitemap
   • WordPress
   • Magento
   • Joomla!
   • Drupal
2. Automatically Create a Sitemap
3. Manually Create a Sitemap

1. Use Your CMS to Generate (or Find) a Sitemap

Since each CMS provider is a little bit different, the most efficient method for creating a sitemap varies depending on which CMS your site uses. While many CMS providers now automatically generate XML Sitemaps, others still require plugins, extensions, or settings configurations to create a sitemap.

Let’s take a look at how to create a sitemap for WordPress.

2. Automatically Generate and Manually Upload a Sitemap

If your website isn’t built on any of the above CMS platforms, you can create and upload your own XML Sitemap with the assistance of website crawling tools. Website crawlers and sitemap generators crawl websites to generate a list of URLs which can be saved as an XML Sitemap file.

The following tools offer free versions and can be used to generate sitemap files for most websites:

XML-sitemaps.com

With XML-sitemaps.com, you just enter your domain URL and click Start. Once the scan is complete, just download the XML sitemap file, upload it to your site’s root, and add it to your /robots.txt file.

The free version of XML-Sitemaps.com will generate an XML Sitemap file with up to 500 pages. The XML-Sitemaps tool gives you the option of upgrading to include more than 500 pages.

Screaming Frog

Screaming Frog is one of the most popular website crawlers. You can crawl up to 500 pages for free. In addition to automatically generating sitemaps, Screaming Frog can be used to export CSV, Google Sheets, and Excel Workbook files.

To generate a sitemap using Screaming Frog, enter your domain URL and click Start. Screaming Frog will begin to crawl your site and generate a list of URLs.

Once the scan is complete, click Sitemaps > XML Sitemaps from the top menu bar.

A pop-up window will appear. We recommend only checking the checkbox next to “PDFs” and leaving the rest unchecked. Click Next.

Your sitemap file should be ready to upload now! Just upload it to your website’s root file and add the sitemap URL to your /robots.txt file.

3. Manually Create and Upload a Sitemap

If your website is small, consisting of only a couple dozen URLs or so, you can manually create a sitemap for your site by following XML Sitemap formatting guidelines. The downside of manually creating a sitemap is that it won’t stay updated as your website changes.  For this reason, we only recommend this option if there isn’t a solution for a dynamic sitemap.

First, gather all your page URLs. For very small sites, you can gather them by following each of your site’s navigation menus/paths down to their deepest level. As you manually gather your URLs, begin grouping them into logical categories. It makes sense to group URLs similarly to how they’re organized in your navigation menus (About, Shopping, Blog, Contact Us, etc.).

Next, code your URLs. You’ll want to use a text editor to create your XML file. If you aren’t familiar with XML, which stands for “extensible markup language,” you can review XML tag definitions and formating standards from sitemaps.org.

Once you’ve coded your URLs, it is time to validate your sitemap. Use a free online tool, like XML-Sitemaps.com’s validator tool, to check that your XML sitemap is properly formatted and operational before adding it to your site.

Finally, upload your XML sitemap file to the root folder of your website and to your robots.txt file, which can be found at the root of your site as well.

For even more in-depth instructions on manually creating a sitemap, read 5 Easy Steps to Creating a Sitemap For a Website from QuickSprout.com…

How to Add Your Website’s Sitemap to RAMP

To start importing and monitoring pages from your website automatically, just add your sitemap path under the RAMP Settings tab. Once you’ve added your sitemap, Accessible Web RAMP will allow you to select either all of your site’s pages or a specific subset you want to prioritize for scanning.

How to add your sitemap to RAMP:

1. Navigate to “Settings” in the lower lefthand corner of RAMP.
2. Select “Automated Scanning” from the horizontal menu bar.
3. Add your sitemap path in the specified field (example: /sitemap.xml).
4. Click “Save” at the bottom of the page.

Issues adding your sitemap? Check out our tips for troubleshooting sitemap issues.

Once you’ve added your sitemap path, your web pages will automatically be imported into RAMP. View the pages you’ve imported from the Pages tab.

When new pages are added to your sitemap, they will also be added to RAMP and scanned for accessibility issues. Accessible Web RAMP will not delete pages that are removed from your sitemap. Similarly, if RAMP automatically adds a page from your sitemap, which you later delete, we will not re-add the page to your RAMP account.

Important Note

If your website has more pages than monthly scans in your plan, RAMP will automatically ingest all your website’s pages but will only scan the first pages until your monthly limit is exhausted. This will happen automatically. If you want more control over which pages are scanned, we recommend importing your web pages via CSV file upload, instead of by linking your sitemap.

Absent or Malformed Sitemap

If your website lacks a sitemap, don’t panic. It is also common for preview/staging environments to lack sitemaps. Instead of using your sitemap, you’ll just need to import pages into your RAMP account either individually or by uploading a CSV file.

Incorrect Domain

If you’ve added your sitemap but RAMP is still unable to scan your pages, check that you’ve entered the correct domain for your website. The domain you entered for your website must match the domain of your sitemap.

If you enter “www.example.com” when your actual domain is “example.com,” RAMP won’t be able to use your sitemap to automatically scan your website. Check the domain you entered to ensure it matches up with your real domain and sitemap.

Extremely Large Sites or Multiple Sitemaps

While Accessible Web RAMP is capable of scanning and monitoring entire websites, sites with an exceedingly large number of pages may not be able to automatically scan.

If you think your site may be too large to scan automatically, contact us for help setting up your account. The upper limit on page scans can be overridden with approval.

Robots.txt Denies Access

Check your Robots.txt file for restrictions or crawl delays that might interfere with RAMP’s ability to scan your sitemap. Sometimes websites will restrict sitemap crawling in their robots.txt file. If your robots.txt restricts access and disallows sitemap crawling, Accessible Web RAMP won’t be able to automatically scan your sitemap.

If your sitemap is unable to scan or you don’t see any scanned pages in your RAMP account within 24 hours, contact our team for assistance setting up your account.

Alternative Methods for Importing Pages

You can also import pages individually by URL path or in bulk by uploading a CSV file containing the sitemap paths of the pages you want to add to RAMP.

Additionally, if you’re having trouble adding pages into Accessible Web RAMP, you can always contact us for complimentary onboarding assistance.

For help putting together and uploading a CSV file, get in touch with us.

If you have a larger website and are trying to use a smaller plan, we recommend adding pages individually to conserve your monthly page scans. Adding pages individually also allows you to prioritize your highest-traffic pages first for scanning.

Complementary Account Setup

Troubleshooting & Frequently Asked Questions

Have questions you aren’t seeing the answers to? Get in contact with us.

• My site currently has an overlay on it that performs DOM manipulation. How does RAMP handle this situation?

  RAMP’s accessibility scanner tries its best to scan a webpage in the exact state that a typical new user will see it in. This means that if you have an overlay on your website, our scanner will attempt to load that overlay and the changes it makes to the HTML. 

  However, due to the way the overlay widgets work we may end up scanning the page before the DOM manipulation occurs. Here are some more technical details about this:

  Knowing when a webpage is fully loaded is actually a difficult problem to solve. For example, a page might “look” fully loaded but actually there is a big gif loading in the background that suddenly pops into view 5 seconds later. Or maybe a webpage is in a continual state of loading, say, with a live chat that’s constantly loading in new chats. Or an analytics service sending pings to and from the browser every few seconds. How can you know which network requests will update the page and which won’t? You mostly can’t! 

  Because of this, our scanner has to draw the line in the sand somewhere and say “this is what a fully loaded web page looks like”. For our scanner, we define a webpage as “loaded” when there are 2 or fewer pending network requests (think loading images, stylesheets, gif, fonts, analytics, data fetching, etc) OR we’ve waited for 2 minutes (subject to change) and the page hasn’t made any significant progress. Once one of those conditions are met, we do our best to test the webpage in whatever state it may be in. 

  It’s important to note that the “final 2″ network requests are typically things like third party analytics or, sometimes, may be your overlay scripts. If overlay scripts are particularly heavy and are consistently some of the last things to load (and execute!!) on your website, you may experience “flaky” testing within RAMP depending on whether that overlay script was fully loaded or not.

• Why am I still not seeing pages from my website show in RAMP?

  If you are not seeing any pages within 24 hours after adding your sitemap to your Website Settings, then there may be a problem scanning your sitemap. Make sure that the sitemap you have specified is accessible to all users (ie. not behind any authentication), and that it contains valid sitemap xml. If you have verified those items, then your site may be blocking our bot. Make sure you don’t have any entries in your robots.txt file that would block our bot from reaching your sitemap. You can learn more about our bot here.

• Why aren’t pages from my website showing up in RAMP?

  Our bot currently scans sites for new pages twice a day, this is subject to change, but may be the reason you aren’t seeing the page yet. If the page still isn’t showing up after 24 hours, make sure that the page is present in the sitemap that you have specified in your Website Settings.

• What is “Active Scanning Job” and how does Accessible Web RAMP schedule scanning jobs?

  “Active Scanning Job” will show the current, automatically scheduled scanning job being processed. Accessible Web RAMP will periodically check your website for pages that need to be scanned based on the what you have specified as their preferred scanning frequency, and when they were last scanned. When it finds one, it will group it with other pages that are also due (or close to due) and scan them together in a “scanning job”. You can expect new pages to be scanned (or at least scheduled in a scanning job) within an hour of them being detected. If you manually request a single page to be scanned, you can expect the scan to complete much quicker than this, likely within 5 minutes.

• What rules are being checked during the accessibility scan?

  Accessible Web uses Axe Core to perform accessibility scans of the pages you request. We run the scan on your page and check for violations of WCAG 2.1 AA by default. If you would like to change the rules that we are checking for, you can adjust these settings in the RAMP Settings tab of you website. We support WCAG 2.0, 2.1 and 2.2 in A, AA and AAA, as well as best practices.

• Is there a way to manually trigger a scan of a page?

  Yes! Navigate to the page within your RAMP account, and click the Scan Page button.