org.apache.lucene.wikipedia.analysis
public class WikipediaTokenizer extends Tokenizer
Modifier and Type | Field and Description |
---|---|
static String |
BOLD |
static String |
BOLD_ITALICS |
static String |
CATEGORY |
static String |
CITATION |
static String |
EXTERNAL_LINK |
static String |
EXTERNAL_LINK_URL |
static String |
HEADING |
static String |
INTERNAL_LINK |
static String |
ITALICS |
static String |
SUB_HEADING |
Constructor and Description |
---|
WikipediaTokenizer(Reader input)
Creates a new instance of the
WikipediaTokenizer . |
public static final String INTERNAL_LINK
public static final String EXTERNAL_LINK
public static final String EXTERNAL_LINK_URL
public static final String CITATION
public static final String CATEGORY
public static final String BOLD
public static final String ITALICS
public static final String BOLD_ITALICS
public static final String HEADING
public static final String SUB_HEADING
public WikipediaTokenizer(Reader input)
WikipediaTokenizer
. Attaches the
input
to a newly created JFlex scanner.input
- The Input Readerpublic Token next(Token result) throws IOException
next
in class TokenStream
IOException
public void reset() throws IOException
reset
in class TokenStream
IOException
public void reset(Reader reader) throws IOException
reset
in class Tokenizer
IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.