archiveis.com

Archiving Single Page Applications (SPAs)

Challenges and solutions for capturing React, Vue, and Angular applications.

The Problem with SPAs

Traditional web archiving tools like wget, curl, or standard Heritrix crawlers operate by sending an HTTP GET request and saving the HTML response. In the early 2000s, this was sufficient.

Today, a modern Single Page Application (SPA) often returns an empty HTML shell:

<!DOCTYPE html>
<html>
<head>
    <script src="/bundle.js"></script>
</head>
<body>
    <div id="root"></div> <!-- Crawler sees nothing here -->
</body>
</html>

If the crawler doesn't execute the JavaScript in bundle.js, it captures a blank screen. The Wayback Machine has historically struggled heavily with this, leading to widespread Link Rot for interactive content.

The Headless Browser Solution

The modern approach to archiving SPAs is using Headless Browsers (like Puppeteer or Playwright). Instead of just downloading the HTML, the archiver spins up a real Chromium instance, loads the page, waits for the network idle state, and then serializes the DOM or captures the network traffic into a WARC file.

Services like archive.today excel at this. They execute the necessary JS and often flatten the resulting page into static HTML, removing the dependency on external API calls.

Webrecorder / Browsertrix

For high-fidelity capturing of complex web apps, Webrecorder is the gold standard. It captures the exact network requests your browser makes while you interact with the page, ensuring that complex state changes and lazy-loaded assets are preserved.

Common Mistakes When Archiving SPAs

  • Stopping too early: Many tools fail because they capture the DOM before the final API calls return data.
  • Missing lazy-loaded assets: Images below the fold are often missed because the headless browser didn't scroll down.
  • Shadow DOM issues: Modern Web Components encapsulate styles and scripts that standard archiving tools struggle to serialize.

SPA Archiving FAQ

Why does an archived React app show a loading spinner forever?

Because the archive captured the JS bundle, but when you view the archived page, the JS tries to fetch live API data (which is either blocked by CORS or the endpoint is dead). The archive needed to capture the API responses, not just the JS bundle.