opennlp.tools.tokenize

Class TokSpanEventStream

java.lang.Object

opennlp.model.AbstractEventStream

opennlp.tools.util.AbstractEventStream<TokenSample>

opennlp.tools.tokenize.TokSpanEventStream

All Implemented Interfaces:

EventStream

public class TokSpanEventStream
extends AbstractEventStream<TokenSample>

This class reads the TokenSamples from the given Iterator and converts the TokenSamples into Events which can be used by the maxent library for training.

Constructor Summary

Constructors

Constructor and Description
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics) Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes the current instance.

Method Summary

Modifier and Type	Method and Description
protected Iterator<Event>	createEvents(TokenSample tokenSample) Adds training events to the event stream for each of the specified tokens.

Methods inherited from class opennlp.tools.util.AbstractEventStream

hasNext, next

Methods inherited from class opennlp.model.AbstractEventStream

remove

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics,
                          Pattern alphaNumeric,
                          TokenContextGenerator cg)

Initializes the current instance.

Parameters:

tokenSamples -

skipAlphaNumerics -

cg -

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics,
                          TokenContextGenerator cg)

Initializes the current instance.

Parameters:

tokenSamples -

skipAlphaNumerics -

cg -

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics)

Initializes the current instance.

Parameters:

tokenSamples -

skipAlphaNumerics -

Method Detail

createEvents

protected Iterator<Event> createEvents(TokenSample tokenSample)

Adds training events to the event stream for each of the specified tokens.

Specified by:

createEvents in class AbstractEventStream<TokenSample>

Parameters:

tokens - character offsets into the specified text.

text - The text of the tokens.

Returns:

an Iterator of training events or an empty Iterator.