in-database supervised model training with multi-framework support
Trains classification and regression models directly within PostgreSQL using pgml.train() SQL function, with bindings to scikit-learn, XGBoost, and LightGBM via pyo3 Python integration layer. Models are persisted in the database as versioned artifacts with automatic hyperparameter tuning and cross-validation, eliminating data movement between application and model servers. The extension uses Rust's pgrx framework to expose these ML operations as native SQL functions that execute within the PostgreSQL process.
Unique: Co-locates training and inference within PostgreSQL using pgrx Rust bindings to Python ML libraries, eliminating network round-trips and data consistency issues inherent in separate model-serving architectures. Models are versioned and stored as first-class database objects with ACID guarantees.
vs alternatives: Faster than cloud ML platforms (SageMaker, Vertex AI) for models under 10GB because data never leaves the database; simpler than MLflow + separate model servers because the database IS the feature store and model registry.
gpu-accelerated embedding generation and semantic search
Generates dense vector embeddings from text using transformer models (BERT, Sentence Transformers, etc.) via pgml.embed() SQL function, with GPU acceleration when available. Embeddings are stored as native PostgreSQL vector columns and indexed using approximate nearest neighbor (ANN) algorithms (HNSW, IVFFlat) for sub-millisecond semantic search. The system uses the Hugging Face Transformers library via pyo3 bindings to load and execute models in-process, avoiding serialization overhead.
Unique: Executes transformer models directly in PostgreSQL process using GPU acceleration, storing embeddings as native vector columns indexed with HNSW/IVFFlat, enabling sub-millisecond semantic search without external vector database. Eliminates round-trip latency and data duplication inherent in separate embedding + vector DB architectures.
vs alternatives: Faster than Pinecone/Weaviate for latency-sensitive applications because embeddings and search happen in-process; cheaper than managed vector DBs because you use existing PostgreSQL infrastructure; simpler than LangChain + external vector DB because the database handles both storage and retrieval.
data preprocessing and feature engineering within sql
Provides SQL functions for common data preprocessing tasks (normalization, encoding, imputation, feature scaling) that execute within PostgreSQL. These functions operate on table columns and return transformed data that can be directly used for model training. The system supports both numeric and categorical transformations, with parameters stored for consistent application during inference.
Unique: Implements preprocessing as native SQL functions that operate on table columns in-place, with transformation parameters stored in the database for reproducible application during inference. Eliminates data movement and ensures preprocessing consistency between training and serving.
vs alternatives: Simpler than Pandas + scikit-learn pipelines because it's a single SQL call; more reproducible than external preprocessing because parameters are stored in the database; faster than exporting data for preprocessing because it happens in-process.
multi-model ensemble and stacking for improved predictions
Combines predictions from multiple trained models using ensemble methods (voting, averaging, stacking) via SQL functions. The system trains meta-models that learn optimal weighting of base model predictions, improving overall accuracy. Ensemble predictions are executed as a single SQL query that calls multiple model inference functions and combines results according to the ensemble strategy.
Unique: Implements ensemble methods as SQL functions that combine multiple model predictions in a single query, with stacking meta-models trained and stored in the database. Ensemble logic is transparent and reproducible because it's defined in SQL.
vs alternatives: Simpler than scikit-learn ensembles because it's a single SQL call; more reproducible than external ensemble code because logic is stored in the database; faster than calling multiple model servers because all inference happens in-process.
time-series forecasting with temporal models
Trains and deploys time-series forecasting models (ARIMA, exponential smoothing, neural networks) using pgml.train() with time-series-specific algorithms. Models learn temporal patterns and seasonality from historical data, then generate future predictions. The system handles time-indexed data, lag features, and rolling window validation automatically. Predictions include confidence intervals for uncertainty quantification.
Unique: Implements time-series forecasting as native SQL functions with automatic lag feature generation and rolling window validation, storing models and predictions in the database. Confidence intervals are generated automatically, enabling uncertainty-aware decision-making.
vs alternatives: Simpler than Prophet or statsmodels because it's a single SQL call; more integrated than external forecasting services because data and models stay in PostgreSQL; faster than cloud forecasting APIs because inference happens locally.
text chunking and preprocessing for rag pipelines
Splits long documents into semantically coherent chunks using pgml.chunk() SQL function with configurable strategies (sliding window, sentence-aware, paragraph-aware). Chunks are stored with metadata (source, offset, chunk_id) and can be directly embedded and indexed for RAG retrieval. The function handles overlapping windows to preserve context across chunk boundaries and supports multiple languages via language-specific tokenizers.
Unique: Implements chunking as a native SQL function within PostgreSQL, preserving chunk-to-source relationships and metadata in the same transaction, enabling end-to-end RAG pipelines without external preprocessing tools. Supports configurable overlap and window strategies to maintain semantic coherence.
vs alternatives: Simpler than LangChain's text splitters because it's a single SQL call; faster than external preprocessing because data doesn't leave the database; maintains referential integrity because chunks are stored as first-class database objects with source tracking.
vector similarity search with approximate nearest neighbor indexing
Performs semantic search using pgvector's native vector type combined with HNSW (Hierarchical Navigable Small World) or IVFFlat approximate nearest neighbor indexes. Queries use cosine similarity, L2 distance, or inner product operators to find k-nearest neighbors in sub-millisecond time. The system automatically manages index creation and tuning parameters (ef_construction, ef_search for HNSW; lists, probes for IVFFlat) based on dataset size.
Unique: Leverages pgvector's native vector type and HNSW/IVFFlat indexes within PostgreSQL, avoiding external vector database overhead. Index parameters are automatically tuned based on dataset characteristics, and search results are returned as standard SQL result sets with full join capability to source data.
vs alternatives: Faster than Pinecone for latency-sensitive applications because search happens in-process; cheaper than managed vector DBs because you use existing PostgreSQL; more flexible than Elasticsearch vector search because you can combine vector similarity with traditional SQL predicates in a single query.
llm inference via openai-compatible api endpoint
Exposes PostgresML as an OpenAI-compatible LLM API server, allowing any client using OpenAI SDK to query models hosted in PostgreSQL. The system supports streaming responses, function calling, and chat completions. Models can be deployed from Hugging Face or custom fine-tuned models, with inference executed on GPU when available. The API layer handles tokenization, prompt formatting, and response streaming without requiring application-level integration changes.
Unique: Implements OpenAI API compatibility layer within PostgreSQL, allowing any OpenAI SDK client to use locally-hosted models without code changes. Inference executes in-process with GPU acceleration, eliminating network latency and API costs while maintaining API surface compatibility.
vs alternatives: Cheaper than OpenAI API for high-volume inference because you pay only for compute, not per-token; faster than cloud APIs for latency-sensitive applications because inference happens locally; more flexible than vLLM because you can combine inference with semantic search and traditional SQL in a single transaction.
+5 more capabilities