May 19, 2026

In honor of block 950,000, which dropped today, we'll be updating our baseline performance numbers and default milestone. This will likely put us over 1hr on the mini and around 5.0 hrs for full validation on the maxi. In an effort to keep the former under 1.0 hrs, I'm looking into the inbound analog to the optimization I made a few months ago, which skips object model construction when emitting full blocks or txs in wire or base16 serialization. This sped up block query a lot, but the inbound analog may significantly improve sync. Specifically, by not parsing a block into the full object model, we can save up to 95% the memory consumption per block, since by default blocks allocate 20x their wire size using the linear accumulator. This is a big performance boost over default allocation, however at a large memory impact. Also the deserialization itself has a modest CPU hit. Presently we obtain the txid and wtxid for each tx directly from the wire buffer. These are attached as cache within each tx object, which significantly optimizes several downstream operations, including witness and non-witness merkle root derivation. In block download we presently perform block::identity checks when under checkpoint/milestone, and full block::check() when validating. However identity is sufficient, and requires not much more from the parse than the tx hashes (also requires the coinbase witness commitment and reservation). By moving block::check to the validation phase (when full block parse is already performed for validation) we retain the simplicity of block check methods, and only have to perform block::identity using parsed hashes and coinbase data. So tonight I'm running a maxi full validation with block::check moved to the validator (full parsing on download still). This will likely not be much of a difference in time. The next step would be to implement the identity checks in network::messages::block (presently they are performed by chain::block methods, where the message object contains the block object). In this design the identity check can be implemented by the message or passed through to the block if its been parsed (and we can implement on-demand deferred parse). The message would carry the moved socket buffer, which is then used to directly populate the store, with zero intermediate allocations. That will require a new query to write the block directly from the wire buffer into the tables. This can be done very efficiently using our rewindable fast stream parser, which is also how we obtain hashes from the buffer presently. So there are a few steps to this. (1) parse the coinbase of witness txs to obtain the reservation and commitment hashes, and (2) move the socket buffer to the block message with a null contained block object, along with parsed tx hashes, (3) expose block identity methods on the block message for node verification, (4) implement direct buffer/hash deserialization to store serialization. But the biggest hurdle I faced in the past was dealing with block::check without a parsed block. The realization that this could move to the validator without a material performance hit makes this a lot easier.