Word2vec

lastmod 2018-08-17created 2017-03-023 edits

Word2vec is a group of related models that are used to produce word embeddings. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic contexts of words. Word2vec takes as its input a large corpus of text and produces a vector space, typically of several hundred dimensions, with each unique word in the corpus being assigned a corresponding vector in the space. Word vectors are positioned in the vector space such that words that share common contexts in the corpus are located in close proximity to one another in the space.

Word2vec was created by a team of researchers led by Tomas Mikolov at Google. The algorithm has been subsequently analysed and explained by other researchers. Embedding vectors created using the Word2vec algorithm have many advantages compared to earlier algorithms like Latent Semantic Analysis.

See also

• Machine learning
• Deep learning

Favorite site

• Wikipedia (en) Word2vec
• Github - facebook research - Pre-trained word vectors (Facebook releases language models pre-trained for 90 languages)
• [Recommended] Summary of word2vec-related theory ¹
• [Recommended] Recent research trends in deep learning-based natural language processing techniques ²
• Deep learning chatbots, PART 1 – INTRODUCTION (Korean translation)
• Deep learning chatbots, PART 2 – IMPLEMENTING A RETRIEVAL-BASED MODEL IN TENSORFLOW (Korean translation)
• SlideShare - Natural language processing techniques in machine learning (I)
• 5-1. Processing natural language with TensorFlow (NLP) – Word Embedding (word2vec)
• Extracting product-related keywords with Word2Vec

References