OpenFlamingo

lastmod 2024-07-02created 2023-04-031 edits

Open-source framework for training and evaluating large multimodal models (LMM)

Features

• Aims to develop a multimodal system that can process image and text input together, like GPT-4
• To that end, an open-source implementation of DeepMind's Flamingo model, an LMM (Large Multimodal Model) that processes and reasons over images, videos, and text
• Included in the first release
  • A Python framework for training Flamingo-style LMMs
  • A large-scale multimodal dataset with interleaved images and text
  • An in-context learning evaluation benchmark for vision-language tasks
  • The OpenFlamingo-9B model, based on LLaMA
• Since Flamingo's training dataset is not public, it was trained on 5 million samples extracted from LAION-2B's Multimodal C4 dataset and 10 million samples

See also

• LMM
• LLaMA

Favorite site

• Announcing OpenFlamingo: An open-source framework for training vision-language models with in-context learning | LAION