Apr 18, 2025

There appears to have been an unexpected performance gain on the mini. I have not tried validating with a large window on the mini in a very long time. Normally I run 5k for validation on the mini, and 50k or higher for milestone. And always 50k or higher on the maxi. This is for data locality, as it improves query response and reduces paging. However the small window also reduces concurrency and can significantly slow milestone for example. Well I accidentally ran a full validation on the mini with a 50k window AND the 100GB Electrum table population AND I had the validation tables set for milestone, so the was only 1 bucket in validate_bk, which turns this 1.7M entry table into a linked list! Nevertheless, the full validation run completed in well under 8 hours - breaking the record. This should mean that the filters could also be populated at the same time, with little impact, since most of that is just what's already available during validation. I'll have to spend some time isolating whether this was caused by the window size increase. But I think that the slew of changes over time made the narrow window less important. If this holds up, we have full validation, with full ElectrumX index, full Compact Filters index, and the always-on full spender, tx, header, and height indexes - completing in under 8 hours on a $350 machine.