1 Enable debugging
in your sites/default/services.yml
twig.config:
debug: true
2 Clear cache
3 enable your browser's HTML comments
in chrome it is something like Preferences > Elements > Show HTML comments
4 find out the template used for this block
by looking in the inspector you can find suggestions on how to override this template
in my case there is a suggestion links--language-block.html.twig
5 copy original template
either from core/modules/system/templates/links.html.twig
or from your themes template directory if you are subtheming
Paste into your theme's template
directory and name it as suggested
6 modify the template
and addClass
to UL
This
<ul{{ attributes }}>
becomes
<ul{{ attributes.addClass("my-class") }}>
7 clear cache again
in your inspector you should see your template and your class
To find out more about twig name conventions and using attributes in templates