Natural language processing
lastmod 2024-12-01created 2016-12-1120 edits
Natural language processing (written in hanja as 自然言語處理 or 自然語處理) refers to natural language understanding, which mechanically analyzes the linguistic phenomena of human speech and turns them into a form computers can understand, or to the whole range of technologies that express such a form again in language humans can understand.
Because it aims to represent language in a form computers can understand, it is very closely related to artificial intelligence, and it is used with nearly the same meaning as computational linguistics, with little distinction.
Category
- Word Embedding (word2vec)
- Stanford CoreNLP
- Stanza (Python)
- Prodigy
- WinkNLP - Developer-friendly natural language processing (NLP) JavaScript library
- Machine translation
- Transformer
Python Libraries
- 10 Best Python Libraries for Natural Language Processing (2024) - Unite.AI
- NLTK - https://www.nltk.org/
- spaCy - https://spacy.io/
- Gensim - https://radimrehurek.com/gensim/
- CoreNLP - https://stanfordnlp.github.io/CoreNLP/
- Pattern - https://github.com/clips/pattern
- TextBlob - https://textblob.readthedocs.io/en/dev/
- PyNLPl - https://pynlpl.readthedocs.io/en/latest/
- scikit-learn
- polyglot
- PyTorch
Morphological analysis
- Kiwi - Intelligent Korean morphological analyzer (Korean Intelligent Word Identifier)
- KoNLPy (Korean: 코엔엘파이) - A Python package for Korean language information processing.
Papers
- Google’s Zero-Label Language Learning Achieves Results Competitive With Supervised Learning (Towards Zero-Label Language Learning) - Training without labels.
See also
Favorite site
- Wikipedia (en) Natural language processing
- Deep Learning for Chatbots, Part 1 – Introduction
- Deep Learning for Chatbots, Part 2 – Implementing a Retrieval-Based Model in Tensorflow
- Performance comparison of Korean morphological analyzers
- Korean meets NLTK and Gensim - PyCon Korea 2015
- [Recommended] A close look at Korean Pre-trained Language Models (2)