Titans Learning to Memorize at Test Time

lastmod 2025-12-10created 2025-12-101 edits

About

• The Titans architecture and the MIRAS framework are designed to let AI models update their core memory even while running, so they can process vast contexts quickly
• Titans combines the speed of RNNs with the accuracy of Transformers, selectively storing input information with high 'surprise' in long-term memory
• MIRAS is a theoretical blueprint that interprets various sequence models in a unified way, systematizing memory structure, bias, forgetting, and the optimization process
• In experiments, Titans and the MIRAS variants (YAAD, MONETA, MEMORA) outperformed state-of-the-art models such as Transformer++ and Mamba-2 in long-context processing and efficiency
• This work shows a shift toward a new generation of long-context AI models that combine the efficiency of RNNs with the expressiveness of Transformers

Documentation

[2501.00663] Titans - Learning to Memorize at Test Time

https://arxiv.org/abs/2501.00663

Favorite site

• Titans + MIRAS: Helping AI have long-term memory
• Google's next-generation deep learning architecture Titans challenges the Transformer stronghold
• Google Titans architecture supports long-term memory formation in AI | GeekNews
• (Paper review) Titans: Learning to Memorize at Test Time - That Time I Got Reincarnated as an AI