It's really interesting to look at the full download/validation/confirmation run in the performance profiler. The building of the prevout table takes .08% of the run. The entire confirmation query takes 1.19% of the run with current optimizations. This is before memory is full, at about 25% of the chain complete. Confirmation is basically free apart from the paging cost.
70% of the run is in script signature validation, 15% is signature hashing (this machine doesn't have shani, otherwise that goes to 3%). 9% is reading blocks and prevouts from the store for validation. 5% is reading blocks off of the wire, hashing, checking and storing.