I installed and configured the Rules module.
When I try to add a rule like notifying a new comment to all users, I do not see the Send mail to all users action.
Is that normal? Is there any other module to install?
Yes it is normal that you do NOT see "Send mail to all users", because there is no such Rules Action in D8 (nor in D7). In D7 there is a Rules Action like "Send an eMail to all users of a selected role", but that one is not (or not yet?) available in D8.
However, within group "System", there is the Rules Action "Send eMail". If you use that one, with "Send to", you can specify 1 eMail address per line. So if you enter the eMail address for each user on a single line, you have a kind of work around for what you're trying to do.
What you probably want to do later on (when the Rules module is more mature), is to use some type list of all your users, and use that list in combination with a "loop" (a typical Rules Action). If this was a D7 question, you could use the Views Rules module to create a view of all your users, and use that for such list.
With that, either you have to be patient a bit, or use the work around suggested above ...