OpenLLMetry

Automatically intercepts and traces LLM API calls (OpenAI, Anthropic, Bedrock, Cohere, etc.) by wrapping provider SDKs at the library level using OpenTelemetry instrumentation hooks, capturing model parameters, prompts, completions, token usage, and latency without requiring manual span creation or code modification. Uses monkey-patching of HTTP clients and SDK methods to inject telemetry collection at runtime.

Instruments LangChain chains, agents, and retrievers and LlamaIndex query engines at the framework abstraction level, creating parent-child span hierarchies that capture the full execution graph including tool calls, retrieval steps, and agent reasoning loops. Uses framework-specific hooks and callbacks to track high-level operations beyond raw API calls.

Captures and versions prompts used in LLM calls with semantic tags and metadata, enabling prompt lineage tracking and A/B testing analysis. Stores prompt versions with associated spans, allowing developers to correlate model outputs with specific prompt versions and identify which prompts produce better results.

Provides an extensible span processor interface that allows developers to implement custom telemetry processing logic (filtering, enrichment, transformation, routing) as pluggable components. Span processors intercept spans before export, enabling custom logic like dynamic sampling, attribute enrichment, backend routing, and data transformation without modifying core instrumentation.

Provides APIs to attach business context metadata (user IDs, session IDs, request IDs, organization IDs) to traces as association properties, enabling correlation of traces with business entities and user sessions. Association properties are propagated through the entire trace tree, allowing observability backends to group and filter traces by business context.

Provides a centralized initialization API (Traceloop.init()) that configures all instrumentation, exporters, and span processors in a single call with environment variable or code-based configuration. Supports batch configuration of multiple instrumentation packages, exporter backends, and privacy controls, reducing boilerplate and enabling environment-specific configuration without code changes.

Automatically instruments vector database operations (Pinecone, Weaviate, Chroma, Milvus) to capture retrieval queries, result counts, similarity scores, and latency. Creates spans for each vector search operation with metadata about query embeddings, filters applied, and results returned, enabling performance analysis of RAG retrieval stages.

Provides Python decorators (@traceloop.workflow, @traceloop.task, @traceloop.agent) to manually wrap custom functions and create spans with automatic context propagation. Decorators capture function arguments, return values, exceptions, and execution time, and automatically associate spans with parent traces through context variables, enabling tracing of application-specific logic beyond instrumented libraries.

Handles OpenTelemetry span lifecycle for streaming LLM responses by deferring span closure until the entire stream completes, capturing partial tokens, stream events, and final completion metrics. Implements custom span processors that buffer streaming events and flush them on stream termination, preventing premature span closure before all data is available.

Provides configurable privacy controls to redact sensitive data from captured spans before export, including prompt/completion masking, PII detection and removal, and selective attribute filtering. Implements span processors that inspect span attributes and events, applying regex patterns and semantic rules to redact or drop sensitive fields while preserving trace structure for debugging.

Exports captured traces, metrics, and events to any OpenTelemetry-compatible backend (Datadog, Honeycomb, Grafana, Jaeger, Traceloop, etc.) using OTLP (OpenTelemetry Protocol) over gRPC or HTTP. Configures exporters and span processors to route telemetry to multiple backends simultaneously, with support for sampling, batching, and retry logic to handle network failures and backend unavailability.

Defines and enforces OpenTelemetry semantic conventions for LLM operations, mapping LLM-specific concepts (model name, token counts, cost, temperature) to standardized span attributes. Implements attribute mappers that normalize data from different LLM providers into consistent attribute names and types, enabling cross-provider querying and analysis in observability backends.

Manages OpenTelemetry context propagation across Python async/await boundaries, thread pools, and concurrent operations using context variables and contextvars. Ensures that spans created in async tasks, thread pool workers, and concurrent operations are correctly associated with parent traces, preventing context loss in complex concurrent workflows.

Collects OpenTelemetry metrics (histograms, counters, gauges) for LLM operations including token counts (input/output), request latency, error rates, and estimated costs. Implements metric exporters that aggregate metrics over time windows and export to observability backends, enabling dashboards and alerts based on LLM usage patterns and costs.