Magma
lastmod 2025-02-23created 2025-02-231 edits
Magma: A Foundation Model for Multimodal AI Agents
A foundation model for multimodal AI agents
About
- Magma is the first foundation model that can interpret multimodal inputs and ground them within its environment, handling complex interactions in both virtual and real worlds
- Goes beyond simple image and video understanding to generate goal-driven visual plans and actions, performing a variety of AI agent tasks
- Achieves state-of-the-art performance on several multimodal tasks, including UI navigation, robot manipulation, and image and video understanding (especially spatial understanding and reasoning)
- Scalable pretraining approach: trained on unlabeled video data together with existing agent data, giving strong generalization and suitability for real-world applications
- Code, model, and UI navigation demo to be released at MSR Forum (2025.02.25).