SeamlessM4T

lastmod 2023-08-24created 2023-08-241 edits

A multimodal foundation model for translation, released by Meta AI

Features

• Meta AI released SeamlessM4T, a foundational multilingual and multitask model that seamlessly translates and transcribes across speech and text
• Supports automatic speech recognition, speech-to-text translation, speech-to-speech translation, text-to-text translation, and text-to-speech translation for many languages
  • Automatic speech recognition for about 100 languages
  • Speech-to-text translation for about 100 input and output languages
  • Speech-to-speech translation supporting about 100 input languages and 35 (+English) output languages
  • Text-to-text translation for about 100 languages
  • Text-to-speech, supporting about 100 input languages and 35 (+English) output languages
• The SeamlessM4T model is released under CC BY-NC 4.0 for researchers and developers
• SeamlessAlign, a 27-hour multimodal translation dataset, was released as well

See also

• Meta

Favorite site

• Introducing a foundational multimodal model for speech translation
• https://ai.meta.com/research/publications/seamless-m4t/
• https://seamless.metademolab.com/
• facebookresearch/seamless_communication
• HuggingFace Demo
• SeamlessM4T: A multimodal foundation model for translation, released by Meta AI - Reading & Info Sharing - PyTorch Korea User Group