Feast

Generates training datasets by performing temporal joins between entity timestamps and feature values, ensuring that only historical feature data available at each training example's timestamp is included. Uses a registry-backed lookup system to resolve feature definitions and executes offline store queries with time-windowed predicates, preventing training-serving skew by guaranteeing models train on the exact feature values that would have been available during inference at that point in time.

Orchestrates scheduled or on-demand jobs that read feature values from offline data sources (data warehouses, data lakes, batch pipelines) and writes them to low-latency online stores (Redis, DynamoDB, PostgreSQL, SQLite) for real-time serving. Uses a Provider abstraction that delegates to compute engines (Spark, Kubernetes, local) and coordinates with the registry to determine which features to materialize, their freshness requirements, and target online store schemas.

Provides a web-based interface for browsing feature definitions, viewing feature statistics, and monitoring materialization jobs. Built with React frontend and Python Flask backend, it queries the registry to display feature schemas, data sources, and lineage. Integrates with feature store to show materialization status and feature freshness metrics.

Abstracts compute engines (Spark, Kubernetes, local Python) behind a unified Provider interface that handles job submission, monitoring, and result retrieval. Providers are responsible for executing materialization jobs, reading from offline stores, and writing to online stores. Supports custom providers for integration with proprietary compute systems (Airflow, Prefect, Dagster).

Defines a type system for entities and features that maps Python types to data warehouse types (int, float, string, timestamp, array, struct). Automatically infers schemas from data sources and validates feature values at materialization and serving time. Supports complex types (arrays, structs) for data warehouses that support them (BigQuery, Snowflake) and serializes them for online stores that don't.

Exposes a feature server (Python, Go, or Java implementation) that accepts entity keys and returns feature values by querying online stores in real-time. The server maintains an in-memory cache of feature definitions from the registry, performs feature lookups with configurable fallback logic (online-to-offline), and supports batch requests for efficiency. Uses protobuf-based request/response schemas for language-agnostic serialization and supports both HTTP REST and gRPC transports.

Maintains a centralized registry (backed by local SQLite, PostgreSQL, or cloud storage) that stores feature definitions, data sources, and metadata as versioned objects. Features are defined as Python classes (FeatureView, StreamFeatureView) with declarative schemas, transformations, and freshness requirements. The registry enables discovery via CLI and SDK, tracks feature lineage, and ensures consistency across training and serving by providing a single source of truth for feature semantics.

Accepts real-time feature updates via HTTP/gRPC push API that writes directly to online stores without requiring batch materialization. Supports both individual feature updates and batch pushes, with configurable schemas and validation. Uses StreamFeatureView definitions to declare streaming features and integrates with Kafka, Kinesis, or custom event sources via connector patterns.

Supports on-demand feature computation via SQL transformations (for data warehouse-native features) and Python transformations (for custom logic). Transformations are defined declaratively in FeatureView definitions and executed at training time (for offline features) or materialization time (for online features). Uses a transformation engine that compiles Python code to SQL when possible (for Spark/BigQuery) or executes Python UDFs for complex logic.

Abstracts offline data sources (Parquet files, data warehouses, data lakes) behind a unified OfflineStore interface that handles schema inference, query compilation, and result retrieval. Supports BigQuery, Snowflake, Spark, DuckDB, PostgreSQL, and Parquet-based stores, allowing teams to switch backends without changing feature definitions. Uses a DataSource abstraction to declare where features are stored and automatically generates appropriate SQL queries for each backend.

Abstracts online feature storage (Redis, DynamoDB, PostgreSQL, SQLite, Cassandra) behind a unified OnlineStore interface that handles schema mapping, serialization, and low-latency lookups. Supports both key-value stores (Redis, DynamoDB) and relational stores (PostgreSQL, SQLite) with automatic schema creation and index management. Uses a consistent key format across stores to enable switching backends without data migration.

Manages Feast configuration via feature_store.yaml files that declare offline stores, online stores, registries, and compute engines. Supports environment-specific overrides (dev, staging, prod) and integrates with Python SDK to load configuration at runtime. Uses a RepoConfig abstraction that validates configuration and initializes store connections, enabling teams to manage infrastructure as code.

Provides command-line interface for common Feast operations: applying feature definitions to registry, materializing features, retrieving training data, and managing online stores. Commands are implemented as Python functions that interact with FeatureStore and Provider abstractions, enabling both interactive development and scripted automation. Supports YAML-based feature definitions and integrates with Python SDK for programmatic access.