I think it's best if your local development environment matches the environment of the servers your site(s) are actually hosted on as much as possible. I've used XAMPP with Drupal and I haven't ever had problems relating to extra libraries being loaded.
That said, if you're looking for a Drupal-optimized solution, Acquia, the company run by the founder of Drupal, provides a Dev Desktop which allows you to set up and host Drupal quickly on Windows or OS X. This works similarly to XAMPP but it's easier to set up. (Warning: Acquia's Drupal distribution is slightly different than the standard Drupal from drupal.org. However, you can easily replace the Acquia Drupal distribution with the one from drupal.org if you want.)
For a different approach, you can try Drupal Quickstart, which is a Virtualbox VM that includes an install of Ubuntu Linux pre-configured with (as per the project page):
- Support for Drupal 7 and Drupal 6
- Ubuntu Linux, Apache, MySQL, APC, XDebug, IMAP, UploadProgress
- Firefox (with developer plugins) and Chrome
- Drush, drush make, and custom drush commands
- Install scripts for: Apache Solr, Aegir, Hudson, and Oracle
- Netbeans and Eclipse, configured with XDebug
- XHProf and Webgrind profilers. Emails logged as files
- cvs, subversion, git, bazaar
This is the "whole development environment in a box" solution.