How do you find broken pages after a website migration?

Last updated: 23 October 2026

Find broken pages by requesting every URL of the old site on the new site, following each redirect to the end, and recording the final HTTP status. Any URL that ends in a 404, a server error, a login wall or a redirect loop is broken, and the old site’s status for the same path tells you whether the migration caused it.

Check it by hand

This method works with a command-line HTTP client such as curl and a spreadsheet.

1. List the old URLs. Take them from the old site’s sitemap.xml (expand a sitemap index into its child sitemaps), the Sitemap: lines in its robots.txt, and a crawl of its links. If the old site is already gone, the new site’s sitemap with its URLs rewritten onto the old host is a fallback, but it only lists pages that still exist. The website migration testing checklist puts this inventory in the context of the whole test.
2. Request each path on both sites. Use the same path on the old host and the new host, and follow redirects hop by hop, so you see every step and not only the end. With curl, curl -sIL prints the status of each hop.
3. Read the final status of each pair.
   • 200 on both: the page exists.
   • 301 or 308, then 200: the page moved and the redirect works. Check that it lands on the matching page and not on the homepage.
   • 404 or 410 on the new site, 200 on the old one: the migration lost the page. Add a redirect or restore it.
   • 401 or 403: a login wall or a firewall. Staging sites often sit behind basic authentication, and bot protection can answer an automated client with 403.
   • 500 to 599: the server failed. Retry once before you log it, since a cold cache can fail the first request.
   • No final status: a redirect loop, a timeout or a refused connection.
4. Note which site is at fault. A page that already answers 404 on the old site is a page the migration did not break. Keep both statuses side by side, for example A: 200 → B: 404, so the fix goes to the right team.
5. Shorten redirect chains. A redirect that passes through two or three hops works, but each hop adds a round trip. Point the old URL straight at its final address.

What DiffiCat automates

DiffiCat runs this check as “Exists”, the first of its three checks, on every page pair.

• The URL list. DiffiCat works through five discovery strategies in order and stops at the first that returns pages: a sitemap URL you supply, the source /sitemap.xml with any sitemap index expanded, the Sitemap: lines in the source robots.txt, the target /sitemap.xml with every URL remapped onto the source, and a crawl of the source homepage. If you supply a sitemap URL, it is the only strategy used.
• Redirects followed hop by hop. Each request follows redirects to the final page and fails on a loop, a timeout or a refused connection.
• Blame indicators. Each failed page names the site at fault, such as A: 200 → B: 404, and says why: not found, forbidden, a server error, a connection that failed, or a request that bot protection blocked.
• Protected staging sites. Behind the lock icon next to each address, you can enter a server password (HTTP basic authentication) or the username and password of the site’s sign-in page.
• Pages worst first. The report leads with a verdict and lists broken pages at the top. A page that fails “Exists” skips the other two checks, since there is nothing to compare.

What this does not catch

• A redirect to the wrong page. A 301 that lands on the homepage answers 200 and passes a status check. In DiffiCat, the “Content” check catches it, because the title and body text no longer match; see checking SEO after a migration.
• A soft 404: a “page not found” message served with status 200. Only a content comparison catches it.
• Pages missing from the list. Discovery finds what the sitemaps and links name; a page that nothing lists is not checked.

DiffiCat has its own limits. Both sites must resolve on the public internet: localhost, loopback and private-network addresses are refused, so deploy staging to a public URL (password protection is fine). Pages are rendered in headless Chromium only. Sign-ins that need a code (MFA), a CAPTCHA or single sign-on are not supported.

Find the broken pages on your sites

Enter your old and new site in the try-it form, or open app.difficat.com and start the free trial: 500 captured pages once when signed in, 100 without signing in, with every check and no card needed.