According to https://sqndr.github.io/d8-theming-guide/libraries/libraries-and-twig.html, you can include a library within twig template to only load it on specific pages using {{ attach_library('theme/library') }}. This appears to be for libraries included in mytheme.libraries.yml.
Is there a way to use this to include a library from sites/all/libraries? It seems silly to add a library to my theme folder when it is already available globally.
There was likely a module that required the files in question to be added to the sites/all/libraries folder. You could review that module on how it adds those files as libraries. Check that module for a .libraries.yml file or check to see if the module implements hook_library_info_build(). Once you know the how to reference the library, you should be able to use the same reference in your twig template.
As an example, the Easychart module (https://www.drupal.org/project/easychart) has both a .libraries.yml and implements the hook in it's .module file. To reference one of those libraries you would use "easychart/easychart.render" or "easychart/lib.highcharts"
/sites/all/libraries, this looks like the folder name the Libraries API module used in D7, which is not ported to D8 yet./librariesby hand. But these libraries can be used only by the module that instructed you to do so, because no one else knows the location of the files.