Uses of Package
opennlp.tools.tokenize
-
Packages that use opennlp.tools.tokenize Package Description opennlp.tools.cmdline.parser opennlp.tools.cmdline.tokenizer opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format.opennlp.tools.formats.ad opennlp.tools.formats.brat opennlp.tools.formats.conllu opennlp.tools.formats.convert opennlp.tools.formats.irishsentencebank opennlp.tools.formats.letsmt opennlp.tools.formats.masc opennlp.tools.formats.muc opennlp.tools.sentdetect Package related to identifying sentece boundries.opennlp.tools.tokenize Contains classes related to finding token or words in a string.opennlp.tools.tokenize.lang opennlp.tools.tokenize.lang.en opennlp.tools.util.featuregen This package contains classes for generating sequence features. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizer Class Description TokenizerEvaluationMonitor TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.brat Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convert Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize Class Description DetokenizationDictionary DetokenizationDictionary.Operation Detokenizer A Detokenizer merges tokens back to their untokenized representation.Detokenizer.DetokenizationOperation This enum contains an operation for every token to merge the tokens together to their detokenized form.SimpleTokenizer Performs tokenization using character classes.TokenContextGenerator Interface forTokenizerMEcontext generators.Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerEvaluationMonitor TokenizerFactory The factory that providesTokenizerdefault implementations and resources.TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer.TokenSample ATokenSampleis text with token spans.WhitespaceTokenizer This tokenizer uses white spaces to tokenize the input text. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang Class Description TokenContextGenerator Interface forTokenizerMEcontext generators. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.