Ideally this implies that as long as download rate is significantly faster than validation rate, download will not materially contribute to total time, and as long as confirmation rate is significantly faster than validation, it will not contribute either. While validation is the long pole, the use of CPU for the other modes reduces availability for the validator, so at that point all CPU optimizations become worthwhile.
(edited)