TinyViT
lastmod 2026-11-10created 2024-07-052 edits
TinyViT is a family of small Vision Transformer models released by Microsoft. It efficiently trains small models through fast pretraining distillation, which precomputes and stores the outputs of a large teacher model and then uses them, and with a hierarchical architecture mixing convolution and attention it achieves high ImageNet accuracy at the scale of a few million parameters. MobileSAM uses this model to replace SAM's image encoder.