You definitely could (Drupal is just a a PHP application, nothing special in that respect). But you'd lose support for stuff like table prefixing.
If you're looking to use PDO, and stay within the bounds of normal Drupal practices, you should install the DBTNG module:
This is a backport of the Drupal 7 PDO database compatibility layer for use with Drupal 6 modules.
This module does not override the existing database functionality but makes the new compatibility layer available under a series of functions identical to their Drupal 7 counterparts named dbtng_ instead of db_. For example with this module, use dbtng_set_active instead of db_set_active, with the exception of the following functions that funciton identically to their Drupal 7 conterparts: db_insert, db_merge, db_update, db_delete, db_truncate, db_select, db_transaction, db_close, db_condition, db_xor, db_and, db_or, db_next_id, and db_like.
That should let you write code that won't need to be changed for Drupal 7 and 8 (unless table/field names change of course).