A grammar-based tokenizer constructed with JavaCC.
See: Description
| Interface Summary | |
|---|---|
| CharStream | This interface describes a character stream that maintains line and column number positions of the characters. |
| StandardTokenizerConstants | |
| Class Summary | |
|---|---|
| FastCharStream | An efficient implementation of JavaCC's CharStream interface. |
| ParseException | This exception is thrown when parse errors are encountered. |
| StandardAnalyzer | Filters {@link StandardTokenizer} with {@link StandardFilter}, {@link LowerCaseFilter} and {@link StopFilter}. |
| StandardFilter | Normalizes tokens extracted with {@link StandardTokenizer}. |
| StandardTokenizer | A grammar-based tokenizer constructed with JavaCC. |
| StandardTokenizerTokenManager | |
| Token | Describes the input token stream. |
| TokenMgrError | |
Note that JavaCC defines lots of public classes, methods and fields that do not need to be public. These clutter the documentation. Sorry.
Note that because JavaCC defines a class named Token, org.apache.lucene.analysis.Token must always be fully qualified in source code in this package.