Oct 31, 2025

Turns out the inflation from wire protocol to json is 2.5x, just based on pulling down some blocks in each format and comparing the file sizes. And even though we don't know the json size before we write it, and we need the buffer to do that, we can use the wire serialization size, which we get for free from the store-deserialized c++ instance of any object. For header this is just a static fixed value. But it varies a lot for others. So a 2.5x buffer estimate will eliminate most iteration over the buffer. Or we can accept a smaller number of iterations to preserve memory. Also resizing down is a cost in terms of reallocations, so there's always a tradeoff.