Trying to do this for a custom entity, but it seems like Domain Access doesn't have its own entity to reference, and I'm not quite sure what method to use. I'm still looking into it, but since I couldn't find a question on it yet, I thought I'd post this ahead of time in case anyone else has done this.
You can override the access handler.
You have to add the handler path in the entity anotation and create an access class extending EntityAccessControlHandler. Inside you can manage the entity access using the rules that you want.
You can copy for learning the Node access Handler: https://api.drupal.org/api/drupal/core%21modules%21node%21src%21NodeAccessControlHandler.php/class/NodeAccessControlHandler/8.2.x
The module https://www.drupal.org/project/domain_entity can be used, the only requirement is that your custom entity is fieldable.