Web archiving
lastmod 2025-08-22created 2024-02-195 edits
Web archiving is the process of collecting portions of the World Wide Web to ensure the information is preserved in an archive for future researchers, historians, and the public. Because of the massive size and amount of information on the web, web archivists typically use web crawlers for automated capture. The largest web archiving organization based on a bulk crawling approach is the Wayback Machine, which strives to maintain an archive of the entire web.
As the share of human culture created and recorded on the web grows, more and more libraries and archives will have to face the challenges of web archiving. National libraries, national archives, and various consortia of organizations are also involved in archiving culturally important web content.
Commercial web archiving software and services are also available to organizations that need to archive their own web content for corporate heritage, regulatory, or legal purposes.
Projects
- ArchiveBox - Self-hosted web archiving tool
- Memex - Full-Text Search in Web History & Bookmarks
- DiskerNet
- LinkAce
- Shiori
- 1History - Browser history backup and visualization tool
See also
- wget
- Archive.today
- Digital archive
- Digital library
- Mirror (computing)
- Virtual artifact
- Wayback Machine
- Korean: 웹 크롤러
- WebCite
- Archiving
- Backup