Feb 18, 2024

I've completed the concurrent header sync implementation. This adds a "chaser" to maintain state across connections, where previously we could only run one connection at a time when syncing headers. The chaser maintains a "currency window" which is set to 60 minutes. Headers older than this are cached in a std lib hash table, along with their validation chain state (which gets stored in the database with each header). When a header chain is both "strong" (more work than the database candidate chain) and "current", the headers from the hash table are moved to the store, and these and any that were not previously in the strong branch are "candidated" (ordered into the candidate chain). This is like block confirmation in the "confirmed" chain, except that the candidate chain is the strongest valid *header* chain available. This is always at least as strong as the confirmed chain and they are usually the same. I just noticed a startup on 10 peers where we went from process startup with empty store to fully-populated and validated candidate chain (831,014 headers) in 17 seconds. Anything under a minute is pretty reasonable given that we don't optimize peer connections (drop slow ones) when syncing headers, as we don't want to prioritize large/fast peers. It's been up and running for a few days with no issues, performing both the initial header sync and ongoing updates in the same logic.