Closeable
, AutoCloseable
@Deprecated public final class IndicTokenizer extends CharTokenizer
StandardTokenizer
instead.AttributeSource.AttributeFactory, AttributeSource.State
Constructor | Description |
---|---|
IndicTokenizer(Version matchVersion,
Reader input) |
Deprecated.
|
IndicTokenizer(Version matchVersion,
AttributeSource.AttributeFactory factory,
Reader input) |
Deprecated.
|
IndicTokenizer(Version matchVersion,
AttributeSource source,
Reader input) |
Deprecated.
|
Modifier and Type | Method | Description |
---|---|---|
protected boolean |
isTokenChar(int c) |
Deprecated.
Returns true iff a codepoint should be included in a token.
|
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
end, incrementToken, isTokenChar, normalize, normalize, reset
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
close, correctOffset
reset
public IndicTokenizer(Version matchVersion, AttributeSource.AttributeFactory factory, Reader input)
public IndicTokenizer(Version matchVersion, AttributeSource source, Reader input)
protected boolean isTokenChar(int c)
CharTokenizer
As of Lucene 3.1 the char based API (CharTokenizer.isTokenChar(char)
and
CharTokenizer.normalize(char)
) has been depreciated in favor of a Unicode 4.0
compatible int based API to support codepoints instead of UTF-16 code
units. Subclasses of CharTokenizer
must not override the char based
methods if a Version
>= 3.1 is passed to the constructor.
NOTE: This method will be marked abstract in Lucene 4.0.
isTokenChar
in class CharTokenizer
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.