News Score: Score the News, Sort the News, Rewrite the Headlines

GlassFlow Load Test: Real-Time Deduplication from Kafka to ClickHouse at Scale

Load Test GlassFlow for ClickHouse: Real-Time Deduplication at Scale By Ashish Bagri, Co-founder & CTO of GlassFlow TL;DR We tested GlassFlow on a real-world deduplication pipeline with Kafka and ClickHouse. It handled 55,00 records/sec published by Kafka and processed 9,000+ records/sec on a MacBook Pro, with sub-0.12ms latency. No crashes, no message loss, no disordering. Even with 20M records and 12 concurrent publishers, it remained robust. Want to try it yourself? The full test setup is ope...

Read more at glassflow.dev

© News Score  score the news, sort the news, rewrite the headlines