org.apache.lucene.analysis
public abstract class Tokenizer extends TokenStream
This is an abstract class.
NOTE: subclasses must override at least one of TokenStream.next()
or TokenStream.next(Token)
.
NOTE: subclasses overriding TokenStream.next(Token)
must
call Token.clear()
.
Modifier and Type | Field and Description |
---|---|
protected Reader |
input
The text source for this Tokenizer.
|
Modifier | Constructor and Description |
---|---|
protected |
Tokenizer()
Construct a tokenizer with null input.
|
protected |
Tokenizer(Reader input)
Construct a token stream processing the given input.
|
protected Reader input
protected Tokenizer()
protected Tokenizer(Reader input)
public void close() throws IOException
close
in class TokenStream
IOException
public void reset(Reader input) throws IOException
IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.