spaCy

Constructs NLP workflows by chaining ordered, stateless processors that sequentially modify immutable Doc objects with linguistic annotations. Each component (tagger, parser, NER, etc.) is declaratively configured in a .cfg file with no hidden defaults, enabling reproducible, version-controlled pipelines that can be easily inspected, modified, and deployed without code changes.

Splits raw text into tokens using language-specific rule sets compiled into the pipeline, handling edge cases like contractions, punctuation, and multi-word expressions without regex overhead. Tokenization is the first pipeline step and produces a Doc object with token boundaries, enabling all downstream components to operate on consistent token boundaries.

Enables training custom NLP models (NER, text classification, dependency parsing, etc.) using declarative .cfg configuration files that specify data paths, hyperparameters, and component settings. Training is reproducible across environments because all settings are explicit in config files, with CLI tools (spacy train, spacy init fill-config) automating setup and validation.

Integrates pretrained transformer models (BERT, RoBERTa, etc.) via the spacy-transformers package, enabling higher accuracy for NER, text classification, dependency parsing, and other tasks. Transformers provide contextualized embeddings that improve accuracy over static word vectors, with GPU acceleration for inference.

Processes large collections of documents efficiently through the pipeline using configurable batch sizes, enabling throughput optimization for information extraction at scale. Batch processing is configured in .cfg files and automatically handles batching during inference, reducing overhead compared to processing documents one-at-a-time.

Provides built-in visualization tools (displacy) for rendering dependency trees, named entities, and other linguistic annotations as interactive HTML or Jupyter notebook visualizations. Enables quick inspection of pipeline output and debugging of NLP models without writing custom visualization code.

Enables developers to write custom NLP components (processors, trainers, evaluators) and register them into the pipeline using a decorator-based API. Custom components receive Doc objects, modify them with annotations, and return them, integrating seamlessly into the declarative pipeline composition model.

Integrates large language models (via spacy-llm package) for few-shot and zero-shot NLP tasks without requiring training data. LLMs are used as components in the pipeline, enabling tasks like entity extraction, text classification, and relation extraction using natural language prompts instead of labeled training data.

Provides pretrained models and language-specific components for 75+ languages, enabling NLP pipelines to process text in diverse languages with language-specific tokenization, POS tagging, parsing, and NER. Language selection is automatic based on model choice or explicit in pipeline configuration.

Assigns grammatical part-of-speech tags (NOUN, VERB, ADJ, etc.) to each token using pretrained statistical models trained on annotated corpora. Supports both traditional statistical taggers and transformer-based models (BERT, etc.) for higher accuracy, with tags stored as immutable annotations on Token objects in the Doc.

Analyzes sentence structure by identifying grammatical relationships between words (subject-verb, object, modifiers, etc.), producing a directed acyclic graph of dependencies. Uses pretrained statistical or transformer-based models to predict head-dependent relationships, enabling extraction of syntactic patterns and semantic role identification.

Identifies and classifies named entities (persons, organizations, locations, products, etc.) in text using pretrained statistical or transformer-based models. Stores entity spans as Span objects with entity labels, enabling downstream filtering, linking, or extraction. Supports training custom NER models on annotated data via the config-based training system.

Classifies entire documents or text spans into predefined categories (sentiment, topic, intent, etc.) using trainable statistical models or transformer-based classifiers. Supports multi-class and multi-label classification, with training via the config-based system and predictions stored as Doc-level attributes.

Reduces words to their base form (lemma) and analyzes morphological features (tense, case, gender, etc.) using pretrained models or rule-based approaches. Lemmatization is language-specific and handles irregular forms, enabling normalization for downstream tasks like information extraction or text mining.

Classifies arbitrary text spans (not just entities) into categories, enabling fine-grained classification of phrases, clauses, or multi-token expressions. Unlike NER which identifies predefined entity types, span categorization allows custom category definitions and overlapping spans, useful for aspect-based sentiment analysis or relation extraction.

Links identified named entities to entries in external knowledge bases (Wikipedia, Wikidata, custom databases) by matching entity mentions to canonical identifiers. Enables knowledge graph construction and entity disambiguation, with linking performed as a pipeline component after NER.

Includes static word embeddings (Word2Vec, GloVe, FastText) for 25+ languages, enabling semantic similarity computation between words, spans, and documents. Vectors are loaded into Doc and Token objects, enabling efficient similarity queries without external vector databases.