Stemming

Stemming is a heuristic process that reduces words to their stems (e.g., running → run), so that inflectional variations do not affect retrieval matching.

Purpose

Makes different word forms match the same stem, improving search recall.

Mechanism

Uses heuristic rules (e.g., the Porter algorithm) to strip suffixes such as ing, ed, s, and ly. Not guaranteed linguistically correct (e.g., universe → univers, university → univers), but fast and deterministic. Snowball is an improved version.

Use cases

Search engine word-form merging
Feature dimensionality reduction for text classification
Query expansion in information retrieval
Word frequency counting

Examples

Input
```
running → run
```
Output
```
Stem run
```
Porter algorithm

Common errors

Over-stemming (e.g., business → bus): Use dictionary-assisted lemmatization

Related terms

Lemmatization is the linguistic process of reducing a word to its dictionary lemma (e.g., better → good); it is more accurate than stemming.

Tokenization splits text into a sequence of semantic units (words, subwords, characters); it is a foundational step in NLP and search engines.

Stop words are high-frequency, low-semantics words (the, is, at, on) that are commonly filtered out in retrieval and NLP to reduce noise.

References