Event Ingestion Pipeline With Dual Write To Postgresql And Clickhouse

via “event ingestion pipeline with dual-write to postgresql and clickhouse”

Open-source LLM observability — tracing, prompt management, evaluation, cost tracking, self-hosted.

Unique: Dual-write architecture separates transactional (PostgreSQL) and analytical (ClickHouse) concerns, enabling real-time trace queries on PostgreSQL while ClickHouse handles aggregations on billions of events. Event enrichment and masking are applied before persistence, ensuring that stored data is clean and compliant.

vs others: More scalable than single-database approaches because ClickHouse columnar storage enables sub-second aggregations on 100M+ events, whereas PostgreSQL would require expensive full-table scans. Dual-write ensures transactional consistency for trace reconstruction while maintaining analytical performance.