That implies it's actually starting to thrash, and that is caused by it drifting too far from validation and download. As it drifts outside the window, it requires more data to be paged even with the very large amount of RAM, and that accelerates as it drifts further.
So what we want is to keep all three lines parallel once blocks are full. (Smaller blocks have a lot more overhead, so it doesn't straighten out fully when plotted by transaction either). This should certainly be possible.
(edited)