Uses of Package
opennlp.tools.tokenize
Packages that use opennlp.tools.tokenize
Package
Description
Experimental package related to converting various corpora to OpenNLP Format.
Experimental package related to the
Arvores Deitadas corpus format.Experimental package related to the corpus format used by the "brat rapid annotation tool" (brat).
Experimental package related to the CoNNL-U format.
Experimental package related to the Irish Sentence Bank format.
Experimental package related to the
letsmt corpus format.Experimental package related to the
MASC corpus format.Experimental package related to the
MUC corpus format.Package related to identifying sentence boundaries.
Contains classes related to finding token or words in a string.
This package contains classes for generating sequence features.
-
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser
-
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizerClassDescriptionA marker interface for evaluating
tokenizers.TheTokenizerModelis the model used by a learnableTokenizer.ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formatsClassDescriptionA
Detokenizermerges tokens back to their detokenized representation.ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.bratClassDescriptionThe interface for tokenizers, which segment a string into its tokens.The
TokenizerModelis the model used by a learnableTokenizer. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convertClassDescriptionA
Detokenizermerges tokens back to their detokenized representation.ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc
-
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc
-
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect
-
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenizeClassDescriptionA
Detokenizermerges tokens back to their detokenized representation.This enum contains an operation for every token to merge the tokens together to their detokenized form.A basicTokenizerimplementation which performs tokenization using character classes.Interface for context generators required forTokenizerME.The interface for tokenizers, which segment a string into its tokens.A marker interface for evaluatingtokenizers.The factory that providesTokenizerdefault implementation and resources.TheTokenizerModelis the model used by a learnableTokenizer.ATokenSampleis text with token spans.A basicTokenizerimplementation which performs tokenization using white spaces. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang
-
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en
-
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen