News Score: Score the News, Sort the News, Rewrite the Headlines

How to build a distributed queue in a single JSON file on object storage

February 12, 2026•Dan Harrison (Engineer)We recently replaced our internal indexing job queue, which notifies indexing nodes to build and update search indexes after data is written to the WAL. The queue is not part of the write path; it's purely a notification system used to schedule asynchronous indexing work. The prior version sharded queues across indexing nodes, so a slow node would block all jobs assigned to it even if other nodes were idle. The new version uses a single queue file on obje...

Read more at turbopuffer.com

© News Score  score the news, sort the news, rewrite the headlines