Don't edit the files in the bootstrap base theme - they will get overwritten when you update to a new version just as you said. Only edit and add files to the subtheme.
About the changes not showing up: Is it possible that you are using the wrong theme hooks or no hooks at all? Just using functions like drupal_add_css() or drupal_add_js() wont work in the template.php unless you call them from a theming hook like template_preprocess_page().
Here is a good tutorial to preprocess functions: Implementing Preprocess and Process Hooks
If you want to generally add a css file to theme (not just to certain templates as you could with the preprocess functions) however, I would suggest adding them in the .info file of your subtheme:
#original bootstrap css files in your subtheme
stylesheets[all][] = css/style.css
stylesheets[all][] = css/jquery.cover.css
#custom sheet
stylesheets[all][] = css/YOURCSS.css
If you intend to use the less files bootstrap ships with hwever, there is no need to even do that: The bootstap subtheme supplies you with a number of less files you can compile into css for your theme using a programm such as crunch. The file you have to compile is the style.less file, which references all other less files. The output is a single style.css file, the one references above. You can either use the less files the subtheme provides or add new ones.