genkit

Genkit implements a language-agnostic action registry system that allows developers to define, compose, and execute flows across JavaScript/TypeScript, Go, and Python SDKs with shared schema validation. Each language SDK maintains a local action registry that can be introspected via a reflection API, enabling cross-language flow composition where actions defined in one language can be orchestrated from another through a standardized message protocol and schema system.

Genkit abstracts model providers (Google AI, Vertex AI, Anthropic, OpenAI, Ollama) behind a unified GenerationRequest/GenerationResponse interface that handles streaming, token counting, and provider-specific features like context caching. The generation pipeline applies middleware at multiple stages (pre-generation, post-generation, model-level) to enable cross-cutting concerns like safety checks, prompt templating, and response transformation without modifying model implementations.

Genkit provides a CLI tool that starts a local development server with a web-based UI for testing flows, actions, and generation calls. The UI displays execution traces, token usage, and allows developers to invoke actions with custom inputs and inspect outputs in real-time. The CLI also manages the telemetry server and provides commands for testing models and running evaluations.

Genkit includes an evaluation framework that defines standard metrics (accuracy, relevance, safety) and allows developers to implement custom evaluators as Genkit actions. Evaluators can be composed into evaluation flows that test generation outputs against expected results, with support for batch evaluation and metric aggregation. The framework integrates with the telemetry system to track evaluation results alongside generation traces.

Genkit supports background execution of long-running model operations (e.g., image generation, video processing) with interrupt and resume capabilities. Developers can submit background jobs that execute asynchronously and poll for results, or implement interrupt handlers to pause execution and resume later with saved state. This enables building applications that handle long-latency operations without blocking the main flow.

Genkit integrates with the Model Context Protocol (MCP) standard, allowing Genkit agents to discover and invoke tools and resources exposed by MCP servers. The framework handles MCP client initialization, tool discovery, and result formatting, enabling seamless integration with MCP-compatible services without custom adapter code.

Genkit provides first-class integration with Firebase (Firestore, Cloud Functions, Cloud Storage) and Google Cloud (Vertex AI, Cloud Run, Cloud Logging) through dedicated plugins. Developers can deploy Genkit flows as Cloud Functions, store data in Firestore, use Vertex AI models, and access Cloud Logging for production observability without manual configuration.

Genkit provides a chat abstraction that manages multi-turn conversation state, including message history, user context, and session metadata. The framework handles message formatting for different model providers, maintains conversation state across turns, and supports session persistence for resuming conversations later. Chat flows can be composed with other Genkit actions to implement complex conversational agents.

Genkit uses a unified schema system (supporting JSON Schema, TypeScript types, Go structs, Python dataclasses) to define expected output shapes for LLM calls. The framework automatically converts schemas to model-specific formats (JSON Schema for OpenAI, Anthropic; protobuf for Vertex AI) and validates responses against the declared schema before returning them to the caller, enabling type-safe structured extraction without manual parsing.

Genkit implements a schema-based function registry where tools are defined with input/output schemas and automatically converted to provider-specific formats (OpenAI function calling, Anthropic tool_use, Vertex AI function calling). The framework handles tool invocation, result formatting, and multi-turn tool use within a single generation call, abstracting away provider differences in tool calling semantics.

Genkit provides a RAG framework with pluggable embedder implementations (Google AI, Vertex AI, Ollama) and vector store integrations (Chroma, Firebase Firestore, custom implementations). The framework handles embedding generation, vector storage, semantic search, and optional reranking in a composable pipeline, allowing developers to swap embedders and vector stores without changing RAG logic.

Genkit instruments all flows, actions, and generation calls with distributed tracing that captures execution traces, latency, token usage, and errors. A built-in telemetry server aggregates traces from multiple SDK instances and exposes them via a reflection API, enabling developers to debug multi-step flows and identify performance bottlenecks without external observability tools.

Genkit provides a prompt templating system that supports variable interpolation, conditional blocks, and tool/function definitions embedded in prompts. Prompts can be composed from multiple templates and passed to generation calls with automatic variable substitution, enabling reusable prompt patterns without string concatenation or manual formatting.

Genkit integrates with model provider context caching features (Vertex AI, Claude) to cache prompt prefixes and reduce latency/cost on repeated requests with the same context. The framework automatically detects cacheable content and applies provider-specific caching strategies without requiring explicit cache management from the developer.

Genkit abstracts multimodal inputs (images, audio, video, code) behind a unified message/part structure that is automatically converted to provider-specific formats. Developers can pass images as URLs, base64, or file paths, and Genkit handles format conversion, media type detection, and provider-specific encoding without manual preprocessing.

Genkit implements a plugin architecture where models, vector stores, embedders, and other components can be registered dynamically at runtime. Plugins are language-specific (JavaScript, Go, Python) and can extend Genkit with custom implementations or integrate third-party services without modifying core framework code. The plugin system uses dependency injection to make plugins discoverable and composable.