org.apache.lucene.analysis
public final class LowerCaseTokenizer extends LetterTokenizer
Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.
Constructor and Description |
---|
LowerCaseTokenizer(Reader in)
Construct a new LowerCaseTokenizer.
|
Modifier and Type | Method and Description |
---|---|
protected char |
normalize(char c)
Collects only characters which satisfy
Character.isLetter(char) . |
isTokenChar
next, reset
next, reset
public LowerCaseTokenizer(Reader in)
protected char normalize(char c)
Character.isLetter(char)
.normalize
in class CharTokenizer
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.