Firebase Genkit

Genkit's core flow system enables developers to compose AI pipelines as strongly-typed, reusable functions with automatic schema validation at each step. Flows are registered in a global action registry and support middleware injection, tracing, and streaming responses. The schema system (leveraging JSON Schema) validates inputs/outputs across all language SDKs (TypeScript, Go, Python), ensuring type safety from definition through execution and enabling reflection-based introspection.

Genkit abstracts multiple LLM providers (Google AI, Vertex AI, Anthropic, OpenAI, Ollama) through a unified GenerateRequest/GenerateResponse interface that normalizes model capabilities. The generation pipeline supports streaming responses via iterators, context caching for expensive prompt prefixes (leveraging provider-native APIs like Claude's prompt caching), and provider-specific part conversions (text, media, tool calls). Middleware can intercept and transform generation requests before reaching the model.

Genkit provides a chat abstraction that manages conversation history and enables multi-turn interactions with LLMs. Chat sessions store messages (user, assistant, tool calls) and support streaming responses. The system handles message serialization, history truncation for context windows, and optional persistence to external storage (Firebase, databases). Chat flows can be composed with tools for agentic conversations.

Genkit can expose flows and tools as an MCP server, enabling external clients (e.g., Claude Desktop, other AI applications) to discover and invoke them. The MCP server implements the Model Context Protocol specification, exposing Genkit actions as MCP resources and tools. This enables Genkit flows to be used by other AI systems without direct integration.

Genkit's middleware system enables intercepting and transforming requests/responses at multiple levels: flow middleware (before/after flow execution), model middleware (before/after LLM calls), and action middleware (before/after any action). Middleware is registered globally or per-action and can modify inputs, outputs, add logging, implement caching, or enforce policies. The middleware chain is composable and supports async operations.

Genkit provides SDKs for TypeScript, Go, and Python that implement a unified API for flows, actions, models, and tools. The SDKs share the same core concepts (action registry, schema validation, middleware) but are implemented in each language's idioms. TypeScript uses decorators and async/await, Go uses interfaces and goroutines, Python uses decorators and async functions. The monorepo structure enables synchronized releases and consistent feature parity.

Genkit provides first-class deployment support for Firebase Cloud Functions and Google Cloud Run, with automatic scaling and integration with Google Cloud services. Flows can be deployed as HTTP endpoints or background functions. The deployment process handles environment configuration, dependency bundling, and observability setup. Genkit automatically configures tracing, logging, and monitoring for deployed functions.

Genkit's dotprompt system provides a YAML-based prompt format that separates prompt definition from code, enabling non-technical users to edit prompts without redeployment. Dotprompt files support Handlebars-style variable interpolation, tool definitions (as JSON Schema), and model configuration (temperature, max_tokens). Prompts are compiled into strongly-typed functions that validate inputs against the declared schema and can be versioned in source control.

Genkit's RAG system provides a modular pipeline for embedding documents, storing them in vector databases, retrieving relevant chunks, and optionally reranking results. The embedder interface abstracts multiple embedding providers (Google AI, Vertex AI, Ollama), while the retriever interface supports pluggable vector stores (Chroma, Firebase Vector Store, custom implementations). Rerankers (e.g., Cohere) can post-process retrieved results to improve relevance. The system handles document chunking, metadata filtering, and hybrid search patterns.

Genkit's tool system enables developers to define functions as JSON Schema and expose them to LLMs for function calling. Tools are registered in the action registry and automatically converted to provider-specific formats (OpenAI function_calling, Anthropic tool_use, Vertex AI tool_calling). The generation pipeline handles tool call parsing, validation, and execution. Tools can be composed into flows and support streaming responses for long-running operations.

Genkit provides built-in tracing for all flows, actions, and model calls through a telemetry system that captures execution traces, latency, token usage, and errors. Traces are sent to a local telemetry server (for development) or cloud backends (Google Cloud Trace, Firebase). The tracing system uses OpenTelemetry-compatible spans and supports custom attributes, logs, and exceptions. The developer UI visualizes traces in real-time, enabling debugging and performance analysis.

Genkit's plugin system enables extending the framework with custom models, vector stores, and integrations. Official plugins include Google AI, Vertex AI (with Gemini, Imagen, Veo models), Firebase (Realtime Database, Firestore, Cloud Storage), Google Cloud (BigQuery, Cloud Run), and safety checks. Plugins register themselves with the action registry and can inject middleware, custom models, and vector store implementations. The plugin architecture supports dynamic loading and configuration through environment variables.

Genkit's evaluation system provides a framework for testing AI-generated outputs against metrics (BLEU, ROUGE, custom evaluators). Evaluators are registered as actions and can be composed into evaluation flows that run against datasets. The system supports batch evaluation, metric aggregation, and comparison across model variants. Evaluation results are stored and can be visualized in the developer UI for performance tracking.

Genkit's developer UI provides a web-based interface for testing flows, viewing execution traces, and inspecting model outputs in real-time. The UI connects to the telemetry server and reflection API to discover available flows, models, and tools. Developers can invoke flows with custom inputs, stream responses, and inspect intermediate results. The UI also displays token usage, latency, and cost estimates for each operation.

Genkit enables LLMs to generate structured outputs (JSON) that conform to a specified JSON Schema. The generation pipeline validates the LLM's output against the schema and automatically parses it into strongly-typed objects. This is implemented via provider-native JSON modes (OpenAI, Anthropic, Google) where available, with fallback to prompt-based guidance for other providers. The schema is used for both validation and documentation.