"all consensus rules" are ALWAYS validated. A milestone validation is based on the assertion by the node operator that a given block is valid. Given that we can safely avoid certain checks that are assured by that assertion. That doesn't include header validation, as otherwise we would be subject to trivial DoS attacks. And header validation is itself trivial cost.
(edited)