Maybe you hope to learn that variable_set is the magical way to store all kinds of structured data in your database, and that variable_get is the way to get it out again. Unfortunately, it is not.
The "persistent variables", as they are often called, are meant for relatively simple key-value pairs that should persist across requests, users and sessions, often until a site admin changes the value. A good example is the site name.
So how does it work?
When Drupal is bootstrapped, all variables are read from the cache (if available) or from the variables table. (See variable_initialize for the exact code.) The variables and stored in the global
$conf variable, a huge array. Drupal does this because it's more efficient than doing a database query every time a variable is needed. (It's also the reason why variable_set is not suitable for large blobs of data: they would be loaded on every request!)
Whenever you use
variable_get('variable_name', 'default_value'), the variable name is looked up in the $conf array. If it exists, the value is returned; else, the default value is returned.
Finally, there is a third member of the family: variable_del. As the name implies, this deletes a variable from the variables table. This function is often used in hook_uninstall implementations, deleting variables when a module is uninstalled.
If you are looking for ways to store more complex custom data structures in the database, check out hook_schema and drupal_write_record, or use the Drupal entity system and Entity API.