Arize Phoenix

Receives distributed traces via gRPC server listening on port 4317 using the OpenTelemetry Line Protocol (OTLP). Spans are parsed from protobuf messages, validated, and persisted to PostgreSQL or SQLite with full trace context preservation including parent-child relationships, attributes, and timing metadata. Supports auto-instrumentation from Python and TypeScript SDKs without code modification.

Exposes traces via Strawberry GraphQL API (src/phoenix/server/api/schema.py) enabling complex queries on span hierarchies, attributes, and relationships. Supports filtering by span kind, status, duration, and custom attributes. Frontend (React/TypeScript in app/) renders interactive trace waterfall diagrams with collapsible span trees, latency heatmaps, and error highlighting. Queries execute against PostgreSQL/SQLite with indexed lookups on trace_id and span_id.

CLI tool (src/phoenix/cli/) provides commands for starting the Phoenix server, exporting traces/datasets to CSV/JSON, and managing database migrations. Supports configuration via environment variables or CLI flags. Enables headless operation for CI/CD pipelines and batch data processing. Export functionality supports filtering by trace ID, span name, or time range.

React/TypeScript frontend (app/) renders traces, datasets, and experiments with interactive UI. Trace viewer displays span waterfall diagrams with collapsible hierarchies, latency heatmaps, and error highlighting. Real-time updates via WebSocket or polling. State management via React hooks and context. Supports dark/light theming. Responsive design for desktop and tablet. Integrates with GraphQL API for data fetching.

Provides Kubernetes deployment manifests (kustomize/) and Helm charts for deploying Phoenix in production. Includes ConfigMaps for configuration, Secrets for API keys, StatefulSets for database, and Deployments for application server. Supports horizontal scaling of the application layer. Health checks and resource limits configured. Documentation for common deployment patterns (single-node, multi-replica, with external PostgreSQL).

Implements authentication via API keys (long-lived tokens for programmatic access) and session tokens (short-lived tokens for web UI). Authorization is role-based (admin, user, viewer) with fine-grained permissions on datasets and experiments. API keys are stored hashed in database. Session tokens are JWT-based with configurable expiration. Supports optional OIDC integration for enterprise SSO.

Python evaluation framework (packages/phoenix-evals/) provides pre-built evaluators for LLM applications: retrieval quality (NDCG, precision@k), hallucination detection, toxicity scoring, and custom LLM-as-judge evaluations. Evaluators are composable functions that accept span data or datasets and return structured scores. Supports both sync and async execution with batching. Integrates with experiment tracking to compare evaluator results across prompt/model variants.

Manages datasets and experiments as first-class objects in Phoenix. Datasets are versioned collections of examples (query, response, reference) stored in the database. Experiments link datasets to prompt/model configurations and store evaluation results. Supports creating datasets from traces, uploading CSV/JSON, and comparing experiment results side-by-side. Experiment tracking stores metadata (model, prompt version, hyperparameters) alongside evaluation scores for reproducibility.

Enables attaching user feedback (ratings, labels, corrections) to spans after they are ingested. Feedback is stored separately from span data and linked via span_id, allowing retroactive annotation without modifying original traces. Supports multiple feedback types: numeric scores (0-5), categorical labels, and free-text corrections. Feedback can be captured via Python client, REST API, or UI. Annotations are queryable and used in evaluation workflows to create ground-truth datasets.

Stores prompts as versioned templates with variable placeholders. Playground interface (internal_docs/specs/playground.md) enables editing prompts, executing them against LLM APIs (OpenAI, Anthropic, Ollama), and comparing outputs. Prompt versions are tracked with metadata (author, timestamp, model). Execution results are stored as traces, enabling evaluation of prompt variants. Supports prompt chaining (multi-step prompts) and parameter sweeping for A/B testing.

Provides language-specific SDKs (arize-phoenix-otel for Python, phoenix-otel for TypeScript) that auto-instrument common libraries (LangChain, LlamaIndex, requests, fetch) without code modification. SDKs register with OpenTelemetry and automatically create spans for LLM calls, database queries, and HTTP requests. Configuration via environment variables or code. Supports both synchronous and asynchronous code paths.

Exposes REST endpoints (src/phoenix/server/api/routes/) alongside GraphQL for programmatic access to traces, datasets, and experiments. OpenAPI schema auto-generated from FastAPI route definitions. Supports CRUD operations on datasets, experiments, and feedback. REST API is simpler than GraphQL for simple queries but less flexible for complex filtering. All endpoints require authentication (API key or session token).

Implements MCP server (js/packages/phoenix-mcp/) enabling Claude and other AI assistants to query Phoenix traces, datasets, and experiments directly. Exposes tools for trace search, dataset creation, and evaluation execution. Assistants can analyze traces, suggest optimizations, and generate evaluation code. MCP server runs as a subprocess and communicates via stdio with the AI assistant.

Abstracts database operations (src/phoenix/server/db/) to support both PostgreSQL and SQLite. Uses SQLAlchemy ORM for schema definition and migrations. Migrations are version-controlled (alembic) enabling schema evolution. Connection pooling and query optimization for PostgreSQL; in-memory SQLite for development. Database schema includes tables for spans, traces, datasets, experiments, evaluations, and feedback with appropriate indexes for common queries.