Language Identification And Script Detection For Multilingual Input

via “language detection and script identification via embedding space geometry”

fill-mask model by undefined. 63,13,411 downloads.

Unique: Language detection emerges from unified multilingual embedding space rather than explicit language classification head; leverages 101-language pretraining to learn language-specific clustering without task-specific architecture

vs others: More efficient than external language detection tools (langdetect, textblob) because reuses existing model inference; produces language embeddings useful for downstream tasks, not just classification