Package opennlp.tools.tokenize
Class TokSpanEventStream
- All Implemented Interfaces:
AutoCloseable,ObjectStream<Event>
-
Constructor Summary
ConstructorsConstructorDescriptionTokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics) Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator. -
Method Summary
Methods inherited from class opennlp.tools.util.AbstractEventStream
close, read, reset
-
Constructor Details
-
TokSpanEventStream
public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.alphaNumeric- A custom alphanumericPatternornull. Default is:"^[A-Za-z0-9]+$", provided byFactory.DEFAULT_ALPHANUMERIC.cg- ATokenContextGeneratorwhich should be used for the event streamd.
-
TokSpanEventStream
public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.cg- ATokenContextGeneratorwhich should be used for the event streamd.
-
TokSpanEventStream
Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.
-