LAION-400M

lastmod 2021-09-15created 2021-09-151 edits

Dataset of 400 million image-text pairs

Features

• The world's largest freely available image dataset
  • Dump of web page crawl data from 2014–2021
• All images/texts filtered with OpenAI's CLIP
  • Image/text pairs with similarity of 0.3 or below filtered out, then manually verified
• Dataset structure
  • 50GB of URL+caption metadata Parquet files
  • 10TB full-version webdataset of 256x256 images/captions/metadata, ready to use for training
  • 1TB of CLIP embeddings for 400M texts/images. Useful for rebuilding KNN indices
  • Two 4GB KNN indices that make searching the dataset easy

See also

• Dataset

Favorite site

• LAION-400-MILLION OPEN DATASET