Mar 24, 2023

Some observations regarding the memory map. My machine is 5 years old, has 256GB RAM, and a new 2TB SSD. As the run was completing I brought up the performance monitor and saw clearly that 255 GB of RAM was allocated to the process. There was a very consistent/flat level of reads that were roughly equal to the amount of writes, though the writes were very spikey. The reads are are expected to hit the hash table headers and walk hash deconfliction linked lists within the files, at arbitrary positions. Given that the store is close to twice my available RAM, this requires a material and very consistent level of memory map paging. This type of store is optimized for a machine with sufficient RAM available to hold the entire blockchain. That was the case when I bought this machine. My mac pro has 1TB RAM, which should make for some good stats. This store ideally targets NVRAM, eliminating paging altogether. But the interface is relational, allowing it to sit on a traditional SQL interface. As the sync completed I watched the SSD go to 100% and the RAM profile ramp down in opposition, until SSD went to 0% and RAM was at about 10%. This was the OS automatically unloading the pages that were no longer in use - as expected.