Apr 26, 2023

Writing is essentially free except for flush, and flush is batch writing, so super efficient. But reading all over the store while you are writing the full store is problematic. This is what can be gleaned from the above. You don't want to force the indexes out of memory as it gets into thrashing. The disk I/O cost of the unguarded sync was basically zero. We want to periodically flush anyway, so by doing so we could presumably manage a very rapid and orderly population of the entire store on multiple threads. The above is just one peer, one thread, all sequential. But there is nothing preventing full concurrency of the population at this point. The write speed of the disk appears to be 500mb/sec, which it hits on a large file copy. So the full store can be flushed in 17 minutes. The time to read the store is bounded by the network bandwidth (not peer or single channel speed).