During install, it is possible to choose the language to be used on the admin screens. However, in this new Drupal 7.40 instance it was not chosen at install time. I need now to change the language from the default English (this is NOT a multilanguage issue).
Every tutorial I read tells me to click the "Add Language" button on Administration > Configuration > Regional and language > Languages. But there is no such button, there isn't even a "Languages" option under "Regional and language":