Mar 13, 2025

I’ve spent some time on design while traveling. In the above test run I extended the tx bucket count from2^30 to 2^31, to increase active header RAM by 4GiB. I also performed a point duplicate search for 1 in 350 points during insert, with the above very successful result. The actual implementation will extend the point hashmap header from 4 byte buckets to 6, with links remaining at 4 bytes and the additional two bytes for identifying buckets with potential duplicates. Since the map is 2^31 buckets, this will increase it from 8GiB to 12GiB, or an increase of 4GiB active headers, as included in the test. However I’ve determined that this level of increase will dramatically reduce the duplicate search rate below the sampled 1/350. Out of 2.5 billion points 1/350 causes 2.5B/350 point hashmap body searches (these are costly and have been the central problem in confirmation all along). That’s stil ~7.7 million searches. However, the adjusted design will call for only about 30,000 searches total. Basically it’s like eliminating the search altogether.