Actually, this is not even limited to milestone, since we write all blocks directly to the store. We only need the deserialization when reading them back out for validation and/or client filters population. So it also provides a significant benefit for validation, as it would consume a huge amount less memory.
(edited)