Jan 1, 2025

So I decided that our method of confirming only blocks that we are validating is probably optimal, yet we need a way to reduce this prevout association cost. This is a hashmap search of each input point (32 byte hash), and a hashmap search of each of these hashes in the tx table, in order to obtain a link to the puts table entry which provides the list of outputs. The links are very fast, but big table hashmap searches are much more expensive, and we avoid them during sync to prevent thrashing. To put it in context, there are 1.03 billion entries in the tx table, and 2.6 billion entries in the input point table (unless compressed but that requires search during sync). This means that we are searching through the entire 2.6 billion entry table and 1.03 billion entry table, 2.6 billion times each. That's 5.2 billion hashmap searches performed during validation, and again during confirmation, if we are validating/confirming the full chain. That's 10.4 billion hashmap searches just to associate all of the previous outputs. It's very fast considering, but can be much better.