Apr 19, 2023

The last quick and dirty test completed. I had staged it for 1/3 of the inputs. Finished up at just over an hour. The rate picked up a bit as blocks got fuller, but was extremely steady and very low disk I/O even with memory full. So I’d expect to see a full run take no more than 3 hrs. Since disk I/O is clearly not a limiting factor (was less than 1mb/sec vs 125mb/sec before), it means that other optimizations, such as replacing the stream and stream reader with a simple memcopy, would probably cut the time down even further.