Microsoft Kosmos-1

lastmod 2023-03-03created 2023-03-035 edits

Microsoft Kosmos-1 - Multimodal LLM(MLLM)

Features

• Multimodal Large Language Model (MLLM) that perceives general modalities, learns in context (few-shot), and follows instructions (zero-shot)
• Model trained on text, images, image-caption pairs, etc., showing impressive performance on the following tasks
  • Language understanding, generation, and NLP without the need for OCR (direct recognition from document images)
  • Multimodal dialogue, image captioning, visual question answering
  • Vision tasks such as image recognition with descriptions (specifying classification via text instructions)
• MLLMs can benefit from cross-modal transfer (transferring knowledge from language to multimodal and from multimodal to language)

Documentations

[2302.14045] Language Is Not All You Need - Aligning Perception with Language Models

https://arxiv.org/abs/2302.14045

See also

• Logic learning machine (LLM)
• Hiring (Microsoft)