TimeCapsuleLLM
lastmod 2026-01-16created 2026-01-161 edits
A large language model trained only on data from 1800–1875
About
- TimeCapsuleLLM is a large language model (LLM) trained only on material from a specific period (1800–1875), aiming to minimize modern bias and reproduce the language and worldview of the time
- The model uses a dataset of historical documents, books, newspapers, legal documents, and more from the London area, reflecting the language style and vocabulary of the era
- Early versions were built on nanoGPT and later versions on Microsoft Phi 1.5, with up to 90GB of data and up to 700M model parameters
- Uses Selective Temporal Training (STT) to train only on data selected from a specific period, designed so that modern concepts are not included
- An experimental project showing the potential of historical language model research and of AI reproducing the language of a given era