Scaling our Observability platform beyond 100 Petabytes by embracing wide events and replacing OTel
Observability at scale: Our internal system grew from 19 PiB to 100 PB of uncompressed logs and from ~40 trillion to 500 trillion rows.
Efficiency breakthrough: We absorbed a 20× surge in event volume using under 10% of the CPU previously needed.
OTel pitfalls: The required parsing and marshalling of events in OpenTelemetry proved a bottleneck and didn’t scale - our custom pipeline addressed this.
Introducing HyperDX: ClickHouse-native observability UI for seamless exploration, correlation, and ...
Read more at clickhouse.com