quite cool! haven't tried it yet, but what's the latency on hot-loading a model? (for instance, loading `stabilityai/stable-diffusion-2-1` for the first API call)
TL;DR: Hugging Face, the NLP research company known for its transformers library (DISCLAIMER: I work at Hugging Face), has just released a new open-source library for ultra-fast & versatile tokenization for NLP neural net models (i.e. converting strings in model input tensors).
Main features:
- Encode 1GB in 20sec
- Provide BPE/Byte-Level-BPE/WordPiece/SentencePiece...
- Compute exhaustive set of outputs (offset mappings, attention masks, special token masks...)
- Written in Rust with bindings for Python and node.js