metawarc

Metawarc is an indexing tool that catalogs WARC collections into a versioned DuckDB database equipped with Parquet sidecars, enabling structured querying, metadata extraction, and payload export. Notable features include support for immutable source archives, atomic publication processes, typed queries accessible via CLI and REST API, and local replay capabilities similar to the Wayback Machine. The tool is designed for efficient analysis and manipulation of web archive datasets, facilitating incremental updates and recovery operations.