Uses of Class
opennlp.tools.tokenize.TokenSample
Packages that use TokenSample
Package
Description
Experimental package related to converting various corpora to OpenNLP Format.
Experimental package related to the
Arvores Deitadas corpus format.Experimental package related to the CoNNL-U format.
Experimental package related to the Irish Sentence Bank format.
Experimental package related to the
MASC corpus format.Contains classes related to finding token or words in a string.
-
Uses of TokenSample in opennlp.tools.cmdline.tokenizer
Methods in opennlp.tools.cmdline.tokenizer with parameters of type TokenSampleModifier and TypeMethodDescriptionvoidDetokenEvaluationErrorListener.misclassified(TokenSample reference, TokenSample prediction) voidTokenEvaluationErrorListener.misclassified(TokenSample reference, TokenSample prediction) -
Uses of TokenSample in opennlp.tools.formats
Methods in opennlp.tools.formats that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.formats.ad
Methods in opennlp.tools.formats.ad that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.formats.conllu
Methods in opennlp.tools.formats.conllu that return TokenSampleMethods in opennlp.tools.formats.conllu that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.formats.convert
Methods in opennlp.tools.formats.convert that return TokenSampleMethods in opennlp.tools.formats.convert that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.formats.irishsentencebank
Methods in opennlp.tools.formats.irishsentencebank that return TokenSampleModifier and TypeMethodDescriptionIrishSentenceBankDocument.IrishSentenceBankSentence.getTokenSample()Methods in opennlp.tools.formats.irishsentencebank that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.formats.masc
Methods in opennlp.tools.formats.masc that return TokenSampleMethods in opennlp.tools.formats.masc that return types with arguments of type TokenSampleModifier and TypeMethodDescription -
Uses of TokenSample in opennlp.tools.tokenize
Methods in opennlp.tools.tokenize that return TokenSampleModifier and TypeMethodDescriptionstatic TokenSampleParses a string sample.protected TokenSampleDetokenizerEvaluator.processSample(TokenSample reference) protected TokenSampleTokenizerEvaluator.processSample(TokenSample reference) TokenizerStream.read()TokenSampleStream.read()Methods in opennlp.tools.tokenize with parameters of type TokenSampleModifier and TypeMethodDescriptionTokSpanEventStream.createEvents(TokenSample tokenSample) Adds training events to the event stream for each of the specifiedsample.protected TokenSampleDetokenizerEvaluator.processSample(TokenSample reference) protected TokenSampleTokenizerEvaluator.processSample(TokenSample reference) Method parameters in opennlp.tools.tokenize with type arguments of type TokenSampleModifier and TypeMethodDescriptionvoidTokenizerCrossValidator.evaluate(ObjectStream<TokenSample> samples, int nFolds) Starts the evaluation.static TokenizerModelTokenizerME.train(ObjectStream<TokenSample> samples, TokenizerFactory factory, TrainingParameters mlParams) Trains a model for theTokenizerME.Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSampleModifierConstructorDescriptionTokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics) Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.WhitespaceTokenStream(ObjectStream<TokenSample> tokens) Initializes aWhitespaceTokenStream. -
Uses of TokenSample in opennlp.tools.tokenize.lang.en
Methods in opennlp.tools.tokenize.lang.en that return TokenSample