org.apache.solr.analysis

Class HyphenationCompoundWordTokenFilterFactory

java.lang.Object

org.apache.solr.analysis.BaseTokenFilterFactory

org.apache.solr.analysis.HyphenationCompoundWordTokenFilterFactory

All Implemented Interfaces:

TokenFilterFactory, ResourceLoaderAware

public class HyphenationCompoundWordTokenFilterFactory
extends BaseTokenFilterFactory
implements ResourceLoaderAware

Factory for HyphenationCompoundWordTokenFilter.

This factory accepts the following parameters:

• hyphenator (mandatory): path to the FOP xml hyphenation pattern. See http://offo.sourceforge.net/hyphenation/.
• encoding (optional): encoding of the xml hyphenation file. defaults to UTF-8.
• dictionary (optional): dictionary of words. defaults to no dictionary.
• minWordSize (optional): minimal word length that gets decomposed. defaults to 5.
• minSubwordSize (optional): minimum length of subwords. defaults to 2.
• maxSubwordSize (optional): maximum length of subwords. defaults to 15.
• onlyLongestMatch (optional): if true, adds only the longest matching subword to the stream. defaults to false.

 <fieldType name="text_hyphncomp" class="solr.TextField" positionIncrementGap="100">
   <analyzer>
     <tokenizer class="solr.WhitespaceTokenizerFactory"/>
     <filter class="solr.HyphenationCompoundWordTokenFilterFactory" hyphenator="hyphenator.xml" encoding="UTF-8"
             dictionary="dictionary.txt" minWordSize="5" minSubwordSize="2" maxSubwordSize="15" onlyLongestMatch="false"/>
   </analyzer>
 </fieldType>

Version:

$Id$

See Also:

HyphenationCompoundWordTokenFilter

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,String>	args The init args
protected org.apache.lucene.util.Version	luceneMatchVersion the luceneVersion arg

Fields inherited from class org.apache.solr.analysis.BaseTokenFilterFactory

log

Constructor Summary

Constructors

Constructor and Description
HyphenationCompoundWordTokenFilterFactory()

Method Summary

Modifier and Type	Method and Description
protected void	assureMatchVersion() this method can be called in the TokenizerFactory.create(java.io.Reader) or TokenFilterFactory.create(org.apache.lucene.analysis.TokenStream) methods, to inform user, that for this factory a luceneMatchVersion is required
org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter	create(org.apache.lucene.analysis.TokenStream input) Transform the specified input TokenStream
Map<String,String>	getArgs()
protected boolean	getBoolean(String name, boolean defaultVal)
protected boolean	getBoolean(String name, boolean defaultVal, boolean useDefault)
protected int	getInt(String name)
protected int	getInt(String name, int defaultVal)
protected int	getInt(String name, int defaultVal, boolean useDefault)
protected org.apache.lucene.analysis.CharArraySet	getSnowballWordSet(ResourceLoader loader, String wordFiles, boolean ignoreCase) same as getWordSet(ResourceLoader, String, boolean), except the input is in snowball format.
protected org.apache.lucene.analysis.CharArraySet	getWordSet(ResourceLoader loader, String wordFiles, boolean ignoreCase)
void	inform(ResourceLoader loader)
void	init(Map<String,String> args) init will be called just once, immediately after creation.
protected void	warnDeprecated(String message)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.solr.analysis.TokenFilterFactory

getArgs

Field Detail

args

protected Map<String,String> args

The init args

luceneMatchVersion

protected org.apache.lucene.util.Version luceneMatchVersion

the luceneVersion arg

Constructor Detail

HyphenationCompoundWordTokenFilterFactory

public HyphenationCompoundWordTokenFilterFactory()

Method Detail

init

public void init(Map<String,String> args)

Description copied from interface: TokenFilterFactory

init will be called just once, immediately after creation.

The args are user-level initialization parameters that may be specified when declaring the factory in the schema.xml

Specified by:

init in interface TokenFilterFactory

inform

public void inform(ResourceLoader loader)

Specified by:

inform in interface ResourceLoaderAware

create

public org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter create(org.apache.lucene.analysis.TokenStream input)

Description copied from interface: TokenFilterFactory

Transform the specified input TokenStream

Specified by:

create in interface TokenFilterFactory

getArgs

public Map<String,String> getArgs()

assureMatchVersion

protected final void assureMatchVersion()

this method can be called in the TokenizerFactory.create(java.io.Reader) or TokenFilterFactory.create(org.apache.lucene.analysis.TokenStream) methods, to inform user, that for this factory a luceneMatchVersion is required

warnDeprecated

protected final void warnDeprecated(String message)

getInt

protected int getInt(String name)

getInt

protected int getInt(String name,
                     int defaultVal)

getInt

protected int getInt(String name,
                     int defaultVal,
                     boolean useDefault)

getBoolean

protected boolean getBoolean(String name,
                             boolean defaultVal)

getBoolean

protected boolean getBoolean(String name,
                             boolean defaultVal,
                             boolean useDefault)

getWordSet

protected org.apache.lucene.analysis.CharArraySet getWordSet(ResourceLoader loader,
                                                             String wordFiles,
                                                             boolean ignoreCase)
                                                      throws IOException

Throws:

IOException

getSnowballWordSet

protected org.apache.lucene.analysis.CharArraySet getSnowballWordSet(ResourceLoader loader,
                                                                     String wordFiles,
                                                                     boolean ignoreCase)
                                                              throws IOException

same as getWordSet(ResourceLoader, String, boolean), except the input is in snowball format.

Throws:

IOException