All node access modules, including Book access, are whitelist only.
Node access modules always GRANT access and never restrict it. (It is a whitelisting rather than a blacklisting system.) If you use two node access modules and one grants access while another does not, access is granted. Exceptions are the Deny access module, as well as the old Module Grants module through which it is possible to combine access grants across multiple modules in a more intuitive way. Without it, the displayed behaviour may appear backwards from what most people would assume, and it's the reason why it is tricky to get involved with multiple node access modules. It is possible to use multiple node access modules in harmony however if for example they are applied to different content types or are giving out different grant types.
This means you need to deny access to books, generally, and then use Book access to grant it per book. If you grant it via Drupal settings, you will grant it to all books, and grants are additive - so some people would be allowed, and some would be "doubly allowed", and no one will be denied.