Jul 18, 2026

When validating we are reading at high concurrency, but this imposes no contention, and the downloader quickly becomes cut off by the validation window (the phases are designed to alternate). But adding capture to validation introduces write concurrency, which is a material part of its cost. I measured that difference today by cutting out the write call, and it's 32% of the capture (without verify) total runtime (1.31 vs. 1.93 hrs). This is the same issue as above. It can be mitigated using thread-static batch tables, which is exactly how our linear memory allocator worked - we just allocate against an array buffer where the array index is an ordinal bound to the thread id. This would just mean that each thread's batch would be either evaluated independently or copied into a single buffer. Not clear which would be optimal. But this would eliminate capture contention.