BERT Pre-Training of Image Transformers

lastmod 2025-02-13created 2024-02-063 edits

BERT is a family of masked language models introduced by Google researchers in 2018. A 2020 literature survey concluded that "in a little over a year, BERT has become a ubiquitous baseline in natural language processing experiments, counting over 150 research publications analyzing and improving the model."

Documentation

[2106.08254] BEiT: BERT Pre-Training of Image Transformers

https://arxiv.org/abs/2106.08254

See also

• ModernBERT - model to replace BERT

Favorite site

• BERT (language model) - Wikipedia
• (Paper review)BEIT : Pre-Training of Image Transformer
• (Paper review) BEiT: BERT Pre-Training of Image Transformers - That Time I Got Reincarnated as an AI