Zstandard's long range mode works wonders for genome sequences without newlines
September 12, 2025 First released with Zstandard 1.3.2 in 2017, the --long range match finder increases the compressor’s search window to at least 128MiB, improving deduplication inside large files. This optional feature had substantial performance overheads at launch, but various optimisations have since brought its performance within shooting distance of Zstandard’s fast defaults. As a fan of Zstandard’s speed and efficiency, I hoped that --long might improve genome compression and bridge the ...
Read more at log.bede.im