Stem Wörter in ihre root form
Stemming reduces Wörter in ihre stem (laufend → ausführen, better → bet). Lernen Porter stemmer und wenn in verwenden lemmatization instead.
- 1
Stemming removes suffixes to find the word stem: "running" → "run", "happily" → "happi", "better" → "bet". The Porter stemmer is the most common algorithm — fast but crude (produces non-words). Lemmatization uses a dictionary to find the lemma: "better" → "good", "running" → "run" — accurate but slower. Use stemming for search indexing, lemmatization for NLP analysis.
- 2 Wortzähler →
Zählen words before and after stemming to verify the vocabulary reduction. Stemming typically reduces vocabulary by 20-40%, which improves search recall (different forms match the same stem).
- 3 Regex-Ersetzung →
Implement a simplified stemmer with regex: remove common suffixes (s, es, ed, ing, ly, ment). Replace (\w+)s$ with $1, (\w+)ing$ with $1. This is crude but demonstrates the concept. For production, use a proper stemmer library (Porter, Lancaster, Snowball).
Häufige Fragen
- Stemming vs lemmatization?
- Stemming is fast and crude — it chops suffixes using rules, producing non-words ("happi" from "happily"). Lemmatization is slower but accurate — it uses a dictionary and morphology to find the actual root word ("happy" from "happily"). Use stemming for search (speed matters, recall is important); use lemmatization for NLP analysis (accuracy matters).
- Which stemmer should I use?
- Porter is the most common — a good balance of speed and accuracy. Lancaster is more aggressive (over-stems). Snowball (Porter2) is an improved Porter. For non-English, use language-specific stemmers (ISnowball, OUD). For production NLP, use lemmatization (spaCy, NLTK) instead.