Is there no current way to get a referencing entity from a view? When clicking on relationship it only shows content referenced from the field, not from the referencing entity.
This is not ready yet:
Implement Entity Reference Backreferences
But you can start from the other side. Start in views with the referencing entity and use a contextual filter with the referenced entity nid and then make the relationsship to get the fields of both entities.