Bag of Words

The bag-of-words model represents text as an unordered word-frequency vector, ignoring grammar and word order; it is the foundational representation for text classification and retrieval.

Purpose

Converts text into fixed-dimensional vectors so that machine learning algorithms can process it.

Mechanism

Build a vocabulary V, count the occurrences of each word in every document, and output an |V|-dimensional vector. Word order and grammar are ignored. Often combined with TF-IDF weighting and stop-word filtering. The dimension equals the vocabulary size.

Use cases

Text classification (Naive Bayes)
Spam detection
Sentiment analysis
Document clustering

Examples

Input

"the cat sat" + "the dog ran"

Output

Word-frequency vector

Ignores word order

Common errors

Ignoring word order loses semantic meaning: Use n-gram or deep models to preserve order

Related terms

TF-IDF (term frequency-inverse document frequency) measures how important a word is to a document: words frequent in the document but rare in the corpus receive high weights.

Tokenization splits text into a sequence of semantic units (words, subwords, characters); it is a foundational step in NLP and search engines.

Cosine Similarity

Cosine similarity measures the cosine of the angle between two vectors, ignoring vector length; it is widely used for text-vector similarity computation.

An N-gram is a sequence of N consecutive units (characters or words) segmented from text, used in language models, similarity, and text analysis.

References