At the moment there is no easy/correct way of doing things as far as I know.
You could try to:
use Commerce Taxonomy Reference (sandbox project). Excerpt from the page:
If you want to add the Contextual Filter 'has taxonomy term ID (with
depth) to a Commerce Product View this module will accomplish that.
This does what you need, but I can't get it to work: there are some versions mismatching etc.
You could move the Taxonomy Reference field away from the Commerce Product to the Product Display node (content type) and use the normal filter (Content: has taxonomy term ID (with depth)).