At the same time if you have many different files to load for each page this will slow down the speed for each page load.
You don't need to worry about this in Drupal, as Drupal will aggregate the CSS files when you have CSS aggregation turned on, and you should always have this enabled on a production site.
As such, you should ensure that only the relevant CSS is attached to an element/page to which it belongs.
For example, I'm creating a site right now that has a booking page. This page has a chart-like schedule on it, showing what time slots are available. It's not unthinkable that this could be swapped out at some time in the future. So I have all the CSS for that chart in a single file, and the chart is attached to that element of the render array. Therefore if anyone ever removes that element, or sets the access on it to FALSE, the CSS will not load.
Drupal will handle the aggregation in the background and serve an aggregated file that contains my file when the user is served the schedule, and serve an aggregated file that does not contain my file, when the user is not served the schedule.
So to sum it up, don't worry about having too many CSS files in Drupal. On the contrary, it's better to have your CSS modular, so that each file contains CSS to a specific element/page, and attach that file to that element/page. Drupal will handle the rest.
And how does the D8 caching system work; will each library be cached
in a separate css file? If I have page X and Y where i only need some
css library on page Y, will both of them share CSS file A and then the
other also have CSS file B? Or will these pages have only 2 different
Aggregates are created as needed. If a page needs file X, Y and Z, it will create an aggregate that contains those files. If another page only needs X and Y, it will create an aggregate with those files. If that same page is visited by a loggeed in user that needs file A, then an aggregate will be created with those files.