Closeable
, AutoCloseable
public final class NGramTokenizer
extends org.apache.lucene.analysis.Tokenizer
Modifier and Type | Field | Description |
---|---|---|
static int |
DEFAULT_MAX_NGRAM_SIZE |
|
static int |
DEFAULT_MIN_NGRAM_SIZE |
Constructor | Description |
---|---|
NGramTokenizer(Reader input) |
Creates NGramTokenizer with default min and max n-grams.
|
NGramTokenizer(Reader input,
int minGram,
int maxGram) |
Creates NGramTokenizer with given min and max n-grams.
|
NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
Reader input,
int minGram,
int maxGram) |
Creates NGramTokenizer with given min and max n-grams.
|
NGramTokenizer(org.apache.lucene.util.AttributeSource source,
Reader input,
int minGram,
int maxGram) |
Creates NGramTokenizer with given min and max n-grams.
|
Modifier and Type | Method | Description |
---|---|---|
void |
end() |
|
boolean |
incrementToken() |
Returns the next token in the stream, or null at EOS.
|
void |
reset() |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public static final int DEFAULT_MIN_NGRAM_SIZE
public static final int DEFAULT_MAX_NGRAM_SIZE
public NGramTokenizer(Reader input, int minGram, int maxGram)
input
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenizer(org.apache.lucene.util.AttributeSource source, Reader input, int minGram, int maxGram)
source
- AttributeSource
to useinput
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory, Reader input, int minGram, int maxGram)
factory
- AttributeSource.AttributeFactory
to useinput
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic boolean incrementToken() throws IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
IOException
public void end()
end
in class org.apache.lucene.analysis.TokenStream
public void reset() throws IOException
reset
in class org.apache.lucene.analysis.TokenStream
IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.