Mar 17, 2025

I believe I have an efficient build of a (generally empty) duplicate table now, which can populate during milestone sync on the mini and very small cost, and allows full validation on the mini in under 10 hours (Core is 15). This is based on runs simulating the point table search load and head size, as it's not implemented yet. But the simulation should be sufficient for modeling the behavior. I've figured out a general technique to reduce the body search cost of false positive duplicate resolution during insert as a function of head size, from (for example) 40% of the full table down to 0.15%, for a load factor 1.5 hashmap, by just doubling the head size and performing some magic in the head template. So for a modest head size increase we can detect all duplicates upon insert, and the time cost is negligible, for milestone and full validation on the mini, around 10%.