If you are married to the idea of running your website via a USB Drive, you can always just download a lamp stack to your computer and configure it to look at usb drive as it's default home directory.
There are several options for this.
- Xampp: General one size fits all lamp stack
- DEV Desktop: A Drupal specific lamp stack
This is great for doing dev work on a duplicate copy of a site for testing in order to not screw up the production. The only problem with this is if you are trying to plug the usb to multiple computers you will have download the software each time and configure it look at the usb for it's root directory.
On the plus side using a local lamp stack you won't lose you ability to edit the site. It'll operate exactly the same way. With the exception that ftp ability will vary. Also it's good practice for doing standard Drupal Migrations.
NOTE: I would suggest if you try these options you operate from a desktop and not a USB. The USB could be for just keeping backups.
If this is something you deem acceptable I can expand on this idea.