May 28, 2024

OK. This is mind blowing!

It'll be fascinating to see how this is impacted when validating and confirming. But, this is simply awesome!

I feel pretty sure that low memory machines will have to validate after sync (because the prevout reads are more costly than the insertion guards that cause thrashing), and certainly higher memory machines will be able to validated concurrently (I've already verified this for the full chain). We can cache block objects in RAM as they come in, and keep the download window small, to prevent re-reading of blocks/txs. But the prevouts would still need to be read.

But low spec machines are more likely to just rely on milestones with background processing of validation. We can support this while allowing the node to be fully functional.

Another run with max machine, no step specified, 3,599 seconds. So a 50k step as no net cost on the run, but significantly improves ordering and reduces cost of validation while downloading. And 81 secs shutdown time, which is awesome for ultra hi RAM full chain sync. After getting to about 8% of the chain Core took 15 minutes to stop on this same machine.

Interesting point about smaller machines using higher milestones and then validatingv the rest after the download. Does that mean we end up creating two types of nodes? Those that validate the entire chain vs those that validate from a large checkpoint? I understand that's the norm anyway right now. It'll be interesting to see how long the validation on a smaller machine takes, even if it has fewer cores. All that aside, the "scaling with hardware" objective is very much happening! 🚀

A milestone is not a checkpoint.

A checkpoint is a soft fork. A milestone is a previously validated block.

The difference is that if the node doesn't see the milestone in the strong headers chain, it doesn't get used. It's only there to bypass validation if the milestone is encountered. It doesn't drive you down a certain path.

So the only question is where do you get it. You can save it from a previous validation, never having to validate that chain again.

So if you want to fire up the node and use it for something other than validating your incoming money, you can have it up and fully usable quickly and validation can continue in the background.

Validation time is analogous to download time, but instead of bandwidth you have CPU capacity: Speed x Threads.

Half the capacity is like half the bandwidth.

So a 64 core machine of 4 MHz cores is 128 times faster validating than a 1 core machine of 2MHz.

I just cranked up the thread count from 16 to 32, after trying several different levels. I was running 16, which was optimal for my 1Gbps network. But 32 is optimal now, and it's running maybe 10% faster. 45-50 minutes?

What's happening is that the things I optimize go away, but the time then gets absorbed waiting on the network, because I'm up against the bandwidth limit. Can only squeeze that grape so much.

But it's really cool to see all of the odd hot spots go away and see the entire report just showing deserialization from the wire and serialization to the disk, with some hashing in the deserialization.

The heat map is getting very clean.

I need to do a run with the memory map sized to the actual tables so that there is no remapping. That will save a few minutes at least.

Because that just stops the network several times, so it can be improved.

I just noticed that SSDs now have r/w speeds of up to 7000MB/s!!

That's 13.2x faster than mine!

Didn't realise that update either. These speeds are going up forever Laura!

1TB PCIe SSD with 5,000MB/s for $89.

2TB version delivered tomorrow for $159

Aw shit, that's PCIe 4/5 and I'm only 1/2/3.

I need a new box 😕

I was looking at data transfer rate, not read/write speed

😁

That thing is 26x faster read/write than my SSD.