Why Gemfury? Push, build, and install  RubyGems npm packages Python packages Maven artifacts PHP packages Go Modules Debian packages RPM packages NuGet packages

Repository URL to install this package:

Details    
tokenizers / tokenizers.pyi
Size: Mime:
# Generated content DO NOT EDIT
from . import (
    AddedToken as AddedToken,
    Encoding as Encoding,
    NormalizedString as NormalizedString,
    PreTokenizedString as PreTokenizedString,
    Regex as Regex,
    Token as Token,
    Tokenizer as Tokenizer,
    __version__ as __version__,
    decoders as decoders,
    models as models,
    normalizers as normalizers,
    pre_tokenizers as pre_tokenizers,
    processors as processors,
    trainers as trainers,
)