You should separate your templating into two logical groups:
Only markup and styling that is relevant to your site theme should go here. You should ask yourself the question - if I was to change my theme - would it affect any functionality of the site? For instance, say you needed to add a block of markup that inserted a third-party analytics tracking. If you put this in
html.tpl.php, it would be removed if you switched theme, breaking site functionality.
If you are augmenting your site with new functionality that requires markup or styling (quite rare) independent of theme, then it should be coupled with the module. The above example of inserting a snippet of JS for a third-party analytics provider is a good one, for instance. Again, ask yourself, what happens if you were to switch the theme. If any site functionality were to break because of a change in theme then it shouldn't be in the theme, but part of the module.
For the most part, your markup and styling will be related to your theme. But say you developed a forum module - you would need some base markup to deliver the functionality; base markup and styling which could then be enhanced by a specific theme, and not break the basics of the forum layout if you decided to implement or switch another theme.