Stem mots en leurs root form
Stemming reduces mots en leurs stem (exécutant → exécuter, better → bet). Apprendre Porter stemmer et quand en utiliser lemmatization instead.
- 1
Stemming removes suffixes to find the word stem: "running" → "run", "happily" → "happi", "better" → "bet". The Porter stemmer is the most common algorithm — fast but crude (produces non-words). Lemmatization uses a dictionary to find the lemma: "better" → "good", "running" → "run" — accurate but slower. Use stemming for search indexing, lemmatization for NLP analysis.
- 2 Compteur de mots →
Compter words before and after stemming to verify the vocabulary reduction. Stemming typically reduces vocabulary by 20-40%, which improves search recall (different forms match the same stem).
- 3 Remplacement par regex →
Implement a simplified stemmer with regex: remove common suffixes (s, es, ed, ing, ly, ment). Replace (\w+)s$ with $1, (\w+)ing$ with $1. This is crude but demonstrates the concept. For production, use a proper stemmer library (Porter, Lancaster, Snowball).
Questions fréquentes
- Stemming vs lemmatization?
- Stemming is fast and crude — it chops suffixes using rules, producing non-words ("happi" from "happily"). Lemmatization is slower but accurate — it uses a dictionary and morphology to find the actual root word ("happy" from "happily"). Use stemming for search (speed matters, recall is important); use lemmatization for NLP analysis (accuracy matters).
- Which stemmer should I use?
- Porter is the most common — a good balance of speed and accuracy. Lancaster is more aggressive (over-stems). Snowball (Porter2) is an improved Porter. For non-English, use language-specific stemmers (ISnowball, OUD). For production NLP, use lemmatization (spaCy, NLTK) instead.