The list that populates the modules page is loaded from the files. But whether a module is enabled or is stored in the database (I assume).
If I enable a module and then delete the files will there be an database entry (to say it's enabled) in the database forever? Or will it be cleared at some time?
Will it slow down the site or clog up my database if I've cleared a lot of modules without disabling them first?
In order to get rid of module's database tables and other contents, you must first disable the module, and then uninstall it. Deleting a module without disabling it is "against the rules", as module_exists('foo') will still return 'true' for these enabled-but-removed modules. Since Drupal tends to call modules only via code that uses the php function function_exists to determine if hooks are available, you usually won't experience any obvious problems if you do this. However, things could get weird if you had a situation where module B uses module_exists() on one of these modules that you removed. The code in question could make assumptions that are only true when the hooks you deleted are executed, which could cause unpredictable behavior. So, in short, don't do that -- always disable and uninstall your modules if you no longer need them.
