I'm not really sure why this works but anyways.. It seems to be what you want.
You'll need Views Contextual Filters OR.
In my test, I used the contextual filter for author UID. So your entity reference field would only shows nodes created by the current user.
- In your view, add the
Content: Author
relationship (don't check required) and Content: Author uid
contextual filter (provide default: current user).
- Enable
Contextual filters OR
in Query settings under Advanced.
- Add a second contextual filter for
Content: Author uid
(provide default: current user)
- Under More at the bottom of the filter, click exclude.
When added this an entity reference field, normal users only see nodes they've created but as admin, I see every node on the site. Weird but it works.