Nov 5, 2024

This was a resounding success, not only in the material reduction to the big box run (.5-1.0 hrs out of 6) but also in the mini run. The mini currently has a validation completion rate of almost 12 hours (plus 1 hours download). This is significantly increased by lack of SHANI, since validation is entirely CPU bound. On the same box, with SHANI implemented, BC takes 15 hours. On a much bigger box without SHANI, Core takes 24 hours - almost doubling the cost. We also have the old libsecp256k1 affecting us here. And our AVX512 implementation doesn't help us because the box doesn't have it. In other words, despite all of that libbitcoin is still about an hour faster than BC if you don't consider confirmation. We should expect BN validation time on mini to be dramatically reduced, possibly in half, to around 6 hours, in an apples-to-apples to BC. On the big box (which is apples-to-apples) BC takes 24.5 hours and BN takes 6.2 (both full validation/confirmation). But confirmation has been the big issue on the mini, due to thrashing from restricted RAM. However, by running confirmation concurrently with and in proximity to validation, the paging cost is dramatically reduced, basically everything is disk-read once instead of twice. [The big box benefits from larger RAM where data read from the network is rarely if ever disk-read during concurrent download/validation/confirmation.] On the mini the independent confirmation completion rate was originally 35 hours (i.e. in addition to the 12 for validation and 1 for download). So end-to-end validation and confirmation was a 47 hour rate. But with concurrent validation/confirmation the combined rate was reduced to under 16.8 hours (17.8 with download). So taking advantage of mmap page locality reduced a full run from ~47 to 18 hours. Considering the lack of SHANI alone, and with no other improvements, this will already outperform BC on the mini. Just applying the BC validation efficiency gained from SNAHI to BN, that would put us at 10.5 hours vs. their 15.