Mar 7, 2025

That approach is to ensure that any duplicates in the point table are recorded in a sidecar table. In a typical run there are none, so everything after that is trivial. There are only 2 costs to doing this, and I did one of them in the first implementation. That's to search each hashmap for the existing item before inserting it. The problem with this is that the performance cost is far worse than the space saved. This was not intended to ensure no duplicates, just to minimize them. A race remains which must be guarded. That's a lock over any search while an insert is pending. That's very, very expensive - basically linearizing all inserts into the point hashmap, which is 2.5 billion entries. But it may still be cheaper than the alternative.