WHAT: Core performance caching stores and serves the whole rendered page with a cache id of the URL. Views caching goes beyond this to cache just the view output.
WHY: It's great for logged in users; other blocks on the page may be more dynamic but the view doesn't have to run a query every time for every user... just the occasional user who generates the page cache upon cache lifetime expire.
SETTINGS: Allowing the "rendered output" to be fresher than the query is useful for things like marking content as new, otherwise just match them up.
BIG PICTURE: Drupal serves page dynamically, using a server to build pages in PHP and access a database (or in-memory cache). This allows awesome features and content editor velocity... but the trade off is needing to understanding caching and do it right.
There's also a lovely Views Content Cache module which clears views cache on content edit. If you want to go further you may want to check out Cache Actions as well, I'd recommend Rules to fine tune this.
Additionally, Blockcache Alter allows you to set "per role", page, user, etc. caching options for each block on your site.
You can also add Page manager and Panels. This allows you to do interesting things like setting the cache "per user", "per role", or a number of other very useful configurations. Though I personally avoid panels.
DOCUMENTATION: For caching and performance in general check out the Drupal.org page.