Bag of Words
lastmod 2019-11-30created 2017-06-212 edits
About
The Bag of Words technique was originally one of the methods for automatically classifying documents, and refers to a technique that determines what kind of document a document is by looking at the distribution of words it contains. For example, if words such as 'exchange rate', 'stock price', and 'interest rate' appear often in a document, it is classified as a document about economics, and if there are many words such as 'backlight', 'exposure', and 'composition', it is classified as a document about photography.
In image processing and computer vision, the Bag of Words technique was mainly used for image categorization or image retrieval, but recently it has also been widely used for recognizing objects and scenes.
Process
The image classification process using the Bag of Words method described above can be summarized as follows.
- Feature Extraction: extract features from the images (SIFT, etc.)
- Clustering: perform clustering on the extracted features to find the codewords, which are the cluster centers. k-means clustering is usually used as the clustering method.
- Codebook Generation: generate a codebook made up of the codewords found. The codebook is not generated for each object class; as in the example in <Figure 2>, only one common codebook is generated, containing the codewords of all classes.
- Image Representation: represent each image as a histogram of codewords. Each image yields one histogram, and the size of this histogram (number of bins) is the same as the size of the codebook (number of codewords making up the codebook).
- Learning and Recognition: BoW-based learning and recognition methods broadly include generative methods using Bayesian probability and discriminative methods using classifiers such as SVM. The Bayesian method classifies objects by interpreting the histogram values for each object class as probabilities, and the discriminative method interprets the histogram values as a feature vector, feeds them into a classifier such as an SVM (support vector machine), and trains it to learn the class boundaries.