Drupal core caching
By default Drupal caches menu items and pages (for anonymous users), but it doesn't cache Views and blocks. Caching these two should both be a high priority.
Custom code quality
Most of the big contrib projects are fairly mature and tuned for high performance. Newer modules might not be. Custom modules, those might not be optimized for performance at all. Typically all the other CDN and reverse proxy (Varnish) solutions that have been mentioned in this post will help you speed up the process of serving static content such as HTML and CSS. But server-side solutions will hardly account for slow code (read: complex, heavy database queries whose results are not cached).
Do you know where your slow query log is? When you launch take a look at it at least once or twice a day. If you can do load testing beforehand even better. The slow query log will tell you what queries in your website you will have to optimize by adding indexes and/or storing the result of those queries in persistent cache using Drupal's native Cache API. Drupal's cache by nature is the database, which is slow compared to Memcached, Memcache and APC. The Cache API acts as an abstraction layer so that you can plug in any of these caching engines without needing to change a single line of your code (other than some configurations in settings.php).
Data-caching wise, the best thing to do is to implement static and persistent in tandem how Jeff from Lullabot outlines. I have stabilized big sites this way.
Also check your Watchdog log for error messages. Each error that goes into the watchdog table is an additional overhead both to the web server's CPU and the database. You want that watchdog log to be clean (or as much as possible).
If you follow this list you should be fine:
Server side
Reverse-proxy (Varnish)
Data cache engine (Memcached, APC, Memcache)
CDN (Akamai, others)
Code Side
Pressflow
Clean watchdog table
Good Cache API implementation, clean slow query log.
Database
Well placed indexes
Do not store unnecessary records, a 100 node database will be always accessed faster than a 3 million node database.