Valohai

Valohai stores ML pipeline definitions and code in Git repositories, automatically tracking complete lineage of experiments including code commits, data versions, parameters, and outputs. The platform integrates with Git workflows to version control pipeline configurations alongside application code, enabling reproducibility by linking each experiment run to specific code commits and dataset versions. This approach eliminates manual experiment logging by capturing the full computational graph at execution time.

Valohai abstracts compute infrastructure through a unified orchestration layer that deploys pipelines to Kubernetes, Slurm HPC clusters, virtual machines, or on-premises data centers without code changes. The platform handles resource allocation, job scheduling, and auto-scaling across heterogeneous infrastructure, allowing teams to run the same pipeline definition on AWS, Azure, GCP, or hybrid environments. This abstraction is achieved through a container-based execution model where pipelines are packaged as Docker containers and submitted to the target infrastructure via Valohai's orchestration API.

Valohai claims to support deploying models for 'batch and real-time inference' but provides no technical documentation on how inference is served, what frameworks are supported, or how models are exposed as APIs. The platform likely packages trained models as containers and deploys them to the same infrastructure (Kubernetes, VMs, Slurm) used for training, but inference serving details including latency, scaling behavior, and API specifications are entirely undocumented. This capability exists but is not production-ready for teams requiring detailed inference specifications.

Valohai automatically captures experiment metadata including metrics, parameters, hyperparameters, and outputs without explicit logging code. The platform provides a web UI for comparing metrics across multiple runs, visualizing performance trends, and querying experiments by tags or parameters. Metrics are stored in a structured format (implementation details undocumented) and indexed for fast retrieval, enabling teams to identify the best-performing model configurations without manual spreadsheet management.

Valohai implements data versioning that avoids storing duplicate copies of datasets by using content-addressable storage or similar deduplication techniques (implementation details undocumented). Teams can tag and query datasets by version, enabling reproducible experiments that reference specific data versions. The platform tracks data lineage through pipelines, showing which datasets were used in which experiments and how data transformations flowed through the pipeline.

Valohai provides a Python SDK that abstracts input/output handling, allowing pipelines to read datasets and write models without hardcoding file paths. The SDK exposes `valohai.inputs()` and `valohai.outputs()` functions that resolve to the correct storage location based on pipeline configuration, enabling the same code to run on different infrastructure (Kubernetes, Slurm, VMs) without modification. This abstraction supports any Python framework (TensorFlow, PyTorch, scikit-learn) and any external library, making Valohai framework-agnostic.

Valohai provides real-time monitoring of compute costs and resource utilization, alerting teams when infrastructure is underutilized (e.g., GPU idle time, unused VM instances). The platform tracks costs across multi-cloud environments and provides visibility into which experiments or pipelines consume the most resources. Cost data is aggregated and presented in a dashboard, enabling teams to optimize spending without manual log analysis.

Valohai provides a Model Hub for tracking and versioning trained models, enabling teams to organize models by project, version, and metadata. The platform supports model handoff between team members by providing a centralized registry where models can be tagged, documented, and promoted through environments (development, staging, production). Model versions are linked to the experiments that produced them, maintaining full traceability from training to deployment.

Valohai supports triggering pipeline execution through webhooks and Git events, enabling automated workflows where code commits, data updates, or external events automatically launch training or inference pipelines. The platform integrates with Git repositories to trigger pipelines on push events, pull request merges, or scheduled intervals. Webhooks allow external systems (data platforms, monitoring tools) to trigger pipelines programmatically, enabling event-driven ML workflows.

Valohai provides audit logging that tracks all actions (experiment runs, model deployments, data access) with timestamps and user attribution, enabling governance and compliance auditing. The platform supports single sign-on (SSO) integration for centralized identity management and role-based access control (RBAC) for restricting who can view, modify, or deploy models. Audit logs are immutable and queryable, supporting compliance requirements like HIPAA or SOX.

Valohai provides pre-built integrations with data platforms (Snowflake, Redshift, BigQuery), NLP libraries (Hugging Face), computer vision frameworks (Super Gradients), and data labeling tools (V7 Labs, Labelbox). These integrations enable pipelines to directly read from data warehouses, pull pre-trained models, and access labeled datasets without custom API code. The platform also supports Docker and Spark for custom integrations, allowing teams to extend beyond pre-built connectors.