The WARC Format (ISO 28500) Explained
Technical specifications of the Web ARChive (WARC) file format used for preserving web resources.
What is a WARC file?
The WARC format (Web ARChive) is a method for combining multiple digital resources into an aggregate archival file together with related information. It is the standard format used by the Internet Archive, Common Crawl, and national libraries worldwide.
Crucially, a WARC file doesn't just save the HTML of a page; it saves the exact HTTP headers that were sent by the server at the exact time of capture. This provides a chain of custody and a historical record of the server configuration.
Anatomy of a WARC Record
A typical WARC file contains three types of records for a single webpage capture:
- warcinfo: Metadata about the crawl itself (who ran it, software used).
- request: The exact HTTP GET request sent by the crawler.
- response: The exact HTTP response received, including headers and the payload (HTML, image, CSS).
WARC vs. PDF
Why not just save pages as PDFs? PDFs are presentation formats; they discard the underlying network reality. A PDF cannot prove when it was taken or where it came from in a machine-readable way. A WARC file contains the cryptographic hashes of the original payload, allowing for verification decades later.
Read more about this in our guide to Legal Citations.
Viewing WARC Files
Because WARC files are raw HTTP records, you cannot simply double-click them to view the website. You need specialized playback software.
| Software | Type | Best For |
|---|---|---|
| ReplayWeb.page | Browser-based | Quickly viewing single WARC files without installing software. |
| pywb (Python Wayback) | Server software | Institutional deployment, running your own Wayback Machine. See Self-Hosting Guide. |
| WARC-Tools | CLI Utilities | Extracting payloads or merging multiple WARC files on the command line. |
WARC Technical FAQ
Does WARC compress data?
Yes, WARC files are almost always compressed using GZIP. The extension is typically .warc.gz. Tools like pywb can read these compressed files directly.
Can I convert a WARC to PDF?
Yes, but it is a one-way trip. You lose the rich network metadata. It is recommended to keep the WARC and generate a PDF derivative if needed for non-technical users.