Have a look at Comment #2 in issue # 2603136 (from the module maintainer). Here is how it looks like (I added the bold markup in it here):
Meaning you can already use Rules to:
- Add a new Group
- Create a new GroupMembership and save it (the equivalent of $group->addMember())
- Add or remove a GroupRole from a GroupType
- React on new GroupMembership or Group entities
- …
What hasn't been done yet is custom Rules actions or conditions. Seeing as 90%+ of Group is pure Entity API CRUD-operations, there has been no custom Rules code yet given how much you can already accomplish out-of-the-box.
Useful Rules we may add could be:
- E-mail all members of a Group, optionally filtered by GroupRole
- Easier-to-understand labels for the above list: "Member joined group" sounds easier than "GroupMembership entity is created"
- Feel free to add here
As per the bold phrase above, I think the exact answer to your question is "Such Rules condition is not available ... yet".
Plan B
However, since you're also using the Privatemsg module, you may want to have a look at the community documentation about Privatemsg-Rules Integration, especially the child page about Introduction to Privatemsg-Rules: it lists the various Rules Events, Conditions and Actions you have available.
More important in the context of this question, is what's explained in the child page "Message notification via email depending on user's role".
See where I'm going ...? No? Well: if you can think of a way to assign a role that corresponds to the name of the group (so 1 role for each role), you have an alternative solution that functionally will do exactly what you want to do. Obviously, 1 role for each "group" is only manageable if you don't have dozens (or hundreds?) of groups ...
Note: I haven't checked it yet, but it may well be possible to use the Rules module to auto-assign such role automatically when a user gets granted access to a group (Homework for you? Or ask a new question?).