MusicLM

lastmod 2023-02-01created 2023-02-013 edits

AI that generates music from text

Features

• Paper published by Google Research
• Generates music highly faithful to the text description, e.g. "a calming violin melody backed by a distorted guitar riff"
• Casts it as a hierarchical sequence-to-sequence modeling task to generate several minutes of 24kHz music
• Far outperforms existing systems in both audio quality and adherence to the text description
• Can also transform melodies in the form of whistling or humming
• Can generate audio in story mode by providing multiple prompts sequentially
• Released the MusicCaps dataset of 5.5k music-text pairs

See also

• Deep learning
• Magenta
• DeepJazz
• BachBot
• FlowMachines
• WaveNet
• GRUV
• MuseNet
• MusicLM - AI that generates music from text
• Riffusion - Stable Diffusion fine-tuned to generate music

Favorite site

• MusicLM web site