Oct 29, 2024

So you spend about 8 years wrestling with that issue, lots of prototyping, making progress in other areas. And finally if you are lucky enough to be on a call with @jungly , it comes to you that you can solve the mmap problem. All you need to do is start over from scratch, redesign the from-scratch store, from scratch again, splitting the headers from the bodies so that you can be truly append only, have true fault detection and recovery, at no material cost to performance. So you build that whole thing and no shit, it really works. So you're back to making that confirmability query perform in constant time per block, because getting rid of that utxo store was the central issue, and this is the resolution. And that works by then there's the mmap paging issue, all over the place. So you work on that, realizing that in the end, old Gordon Moore is coming to the rescue. With the full chain in RAM, all queries are stupid fast, and you can do things like download, parse, hash, check, index and store 350,000 txs/sec on a $350 mini pc.