Nov 4, 2024

This was a resounding success, not only in the material reduction to the big box run (.5-1.0 hrs out of 6) but also in the mini run. The mini currently has a validation completion rate of almost 12 hours (plus 1 hours download). This is significantly increased by lack of SHANI, since validation is entirely CPU bound. On the same box, with SHANI implemented, BC takes 15 hours. On a much bigger box without SHANI, Core takes 24 hours - almost doubling the cost. We also have the old libsecp256k1 affecting us here. And our AVX512 implementation doesn't help us because the box doesn't have it. In other words, despite all of that libbitcoin is still about an hour faster than BC if you don't consider confirmation. We should expect BN validation time on mini to be dramatically reduced, possibly in half, to around 6 hours, in an apples-to-apples to BC. On the big box (which is apples-to-apples) BC takes 24.5 hours and BN takes 6.2 (both full validation/confirmation). But confirmation has been the big issue on the mini, due to thrashing from restricted RAM. However, by running confirmation concurrently with and in proximity to validation, the paging cost is dramatically reduced, basically everything is disk-read once instead of twice. [The big box benefits from larger RAM where data read from the network is rarely if ever disk-read during concurrent download/validation/confirmation.] On the mini the independent confirmation completion rate was originally 35 hours (i.e. in addition to the 12 for validation and 1 for download). So end-to-end validation and confirmation was a 47 hour rate. But with concurrent validation/confirmation the combined rate was reduced to under 16.8 hours (17.8 with download). So taking advantage of mmap page locality reduced a full run from ~47 to 18 hours. Considering the lack of SHANI alone, and with no other improvements, this will already outperform BC on the mini. Just applying the BC validation efficiency gained from SNAHI to BN, that would put us at 10.5 hours vs. their 15.

There is an assumption from what I've seen on Delving that we are just speeding up sync using concurrency, but due to the lack of a utxo store we will be slower at validation. The numbers we are seeing so far paint a very different picture. We are 4 times faster on a big box fully-validating run in an apples-to-apples comparison, and may be almost twice as fast on a mini run. That certainly takes advantage of concurrency, but validation is clearly very performant. The big cost for us is paging off the disk, but with single block-at-a-time compact block validations the relevant data will be mostly if not entirely already in RAM - and we don't have to construct a block, validate txs, or store any utxos. I suspect that, especially for miners with big boxes, we will be much faster there as well.