langfuse

Captures LLM interaction traces across heterogeneous SDKs (Langchain, LiteLLM, OpenAI SDK, LlamaIndex) via unified ingestion API endpoints that normalize events into a PostgreSQL-backed trace graph. Uses event enrichment and masking pipelines to standardize observations (LLM calls, retrievals, tool executions) into parent-child relationships, enabling full execution path reconstruction without modifying user application code.

Accepts OpenTelemetry Protocol (OTLP) traces via gRPC/HTTP endpoints and maps OTel semantic conventions (span attributes, events, status codes) to Langfuse trace domain model (observations, scores, metadata). Implements dual-write architecture to PostgreSQL and ClickHouse for real-time querying and historical analytics, with automatic schema validation and attribute masking for PII.

Supports batch operations on multiple traces (export, delete, tag, score, assign to dataset) via async job queue with progress tracking and error recovery. Uses Redis-backed job queue for reliable processing with automatic retry logic and dead-letter queue for failed jobs. Implements batch selection UI with checkbox filtering and action confirmation, supporting 1k+ trace selections without UI blocking.

Implements configurable data retention policies at project level, automatically archiving or deleting traces based on age, cost, or custom criteria. Uses background scheduled jobs to enforce retention policies without manual intervention. Supports tiered storage (hot PostgreSQL, cold ClickHouse, archive S3) with automatic data migration based on retention tier. Provides audit trail of deleted traces for compliance.

Streams new traces to connected clients via WebSocket or Server-Sent Events (SSE), enabling live dashboard updates without polling. Implements efficient delta updates (only changed fields) to minimize bandwidth. Uses tRPC subscriptions for real-time updates with automatic reconnection and backpressure handling. Supports filtering live streams by project, trace status, or custom criteria.

Executes automated evaluations on captured traces using LLM-as-Judge pattern via Redis-backed job queue (evalExecutionQueue, llmAsJudgeExecutionQueue). Supports configurable scoring rubrics with multi-step evaluation logic, integrates with OpenAI/Anthropic/custom LLM providers for judgment, and stores scores as observations linked to traces. Uses background worker processes to parallelize evaluation across multiple traces with configurable retry logic and error handling.

Implements multi-tenant isolation via project-scoped API keys and role-based access control (RBAC) with configurable permissions per user role. Supports SSO integration (OIDC, SAML) for enterprise deployments and API key management with automatic rotation and scoping. Uses tRPC internal API with authentication middleware and PostgreSQL-backed permission checks to enforce access control across all endpoints.

Stores prompt templates with version control, enabling side-by-side comparison of prompt variants via experiment framework. Integrates with trace capture to automatically tag observations with prompt version and experiment ID, enabling statistical analysis of prompt performance. Uses PostgreSQL for prompt storage and ClickHouse for aggregated experiment metrics (success rate, latency, cost per variant).

Web-based playground for testing LLM calls with live model switching across OpenAI, Anthropic, Ollama, and custom endpoints. Supports prompt templating with variable substitution, message history management, and parameter tuning (temperature, top_p, max_tokens). Captures all playground interactions as traces for debugging and evaluation, with side-by-side model comparison and response streaming.

Manages datasets of LLM inputs/outputs with annotation queue system for human review and labeling. Supports batch creation from captured traces, manual annotation workflows with configurable label schemas, and export to training/evaluation formats. Uses PostgreSQL for dataset storage and annotation state management, with optional LLM-assisted annotation suggestions via LLM-as-Judge pattern.

Groups related traces into sessions representing multi-turn conversations or user interactions. Automatically links observations across turns using session ID, preserving conversation context for debugging and analysis. Supports session-level metrics (total cost, latency, user satisfaction) and filtering by session properties (user_id, environment, model). Uses PostgreSQL for session storage and ClickHouse for session-level aggregations.

Automatically calculates and aggregates LLM costs based on token usage and configurable pricing models for OpenAI, Anthropic, and other providers. Stores token counts (input, output, total) per observation and aggregates costs at trace, session, and project levels. Uses ClickHouse for time-series cost analytics and cost trend analysis. Supports custom pricing models for fine-tuned models or enterprise pricing agreements.

Provides advanced filtering and search across captured traces using PostgreSQL full-text search and ClickHouse analytics queries. Supports complex filter combinations (trace status, model, cost range, latency, user properties) with saved views for reusable filter sets. Implements virtualized table rendering for efficient display of 10k+ traces with sorting, pagination, and batch actions. Uses tRPC internal API for filter execution and ClickHouse for aggregated analytics (histograms, percentiles, distributions).