rut5-base-summ

Implements a T5-base encoder-decoder transformer (220M parameters) fine-tuned on multilingual summarization datasets including Russian dialogue (SAMSum-RU, RuDialogSum), news articles (Gazeta, MLSUM), and Wikipedia abstracts (Wiki Lingua). Uses teacher-forcing during training and beam search decoding at inference to generate abstractive summaries that preserve semantic content while reducing length. Supports both Russian and English input with language-agnostic token embeddings learned during multi-dataset training.

Model trained on heterogeneous summarization datasets (dialogue, news, Wikipedia) using curriculum learning or mixed-batch training, allowing it to generalize across domains without catastrophic forgetting. The T5 architecture's text-to-text framework treats all summarization tasks uniformly (input: 'summarize: [text]', output: '[summary]'), enabling zero-shot transfer to new domains via prompt engineering or light fine-tuning on domain-specific data.

Generates summaries using beam search (not greedy decoding), maintaining multiple hypotheses during generation and selecting the highest-scoring sequence according to a scoring function that balances log-probability with length penalties. Supports configurable beam width (typically 4-8), length normalization to prevent bias toward short outputs, and early stopping when all beams have generated end-of-sequence tokens. Implemented via transformers library's generation utilities with native support for batched inference.

Model weights stored in SafeTensors format (a safer, faster alternative to PyTorch's pickle-based .pt files) enabling single-file loading without arbitrary code execution. SafeTensors uses memory-mapped I/O, reducing peak memory usage during model loading and enabling lazy loading of individual weight tensors. Checkpoint includes full tokenizer configuration (vocabulary, special tokens) for seamless integration with transformers pipeline API.

Model is compatible with Hugging Face's managed Inference Endpoints service, enabling one-click deployment without managing infrastructure. Endpoints service automatically handles model loading, batching, scaling, and provides a REST API (with optional authentication) for inference. Supports both CPU and GPU hardware selection, with automatic scaling based on request volume. Integrates with transformers library's pipeline API for standardized input/output handling.

Includes a trained SentencePiece tokenizer (32K vocabulary) optimized for Russian and English text, with special tokens for task prefixes ('summarize:', 'translate:'), padding, and unknown tokens. Tokenizer handles subword segmentation, preserving Russian morphology better than character-level approaches. Transformers library's AutoTokenizer API automatically loads the correct tokenizer configuration from the model card, ensuring input/output alignment without manual token ID mapping.

Model trained on both Russian and English datasets (SAMSum-RU for Russian dialogue, SAMSum for English dialogue, MLSUM for news in both languages) enables zero-shot summarization of English text without English-specific fine-tuning. T5's multilingual token embeddings learn shared semantic representations across languages, allowing knowledge from Russian training data to transfer to English inputs. No language detection or routing logic required; model handles both languages via unified input format.