Jun 6, 2026

If the above transformation is indeed necessary, then it would be possible for us to produce the signature batches in this vertical format, however it requires either (1) making an up-front assumption about the chunk size (i.e. how many signatures we expect to batch in one go), or (2) constructing 4 independent mmaps per logical table (which would result in 16 tables total). Both of these are feasible. It should be that they are close to equivalent from an I/O cost standpoint. #2 is more flexible though it requires a little more setup overhead and some amount of increased contention.