Fortunately I got the answer, by looking at the solution described in a comment to View of products with multi-value product reference field show multiple rows per product node. Here is a summary of it:
I created a view of product display nodes. Added the relationship to products. Added two fields "Content: Title" and "Commerce Product: Image".
Got a row for every display node and product (so a node that references two products gets two rows). This is what I expect to happen.
Possible solution A:
I click on "Query settings" in the third column, and in query settings enable "Distinct" (it's a checkbox).
That kills the duplicates.
However, DISTINCT is usually a bit stupid, so it might not work in all cases. In those cases, you can try the next possible solution.
Possible solution B:
I enable Aggregation (again, it's in the third column). I make sure to have a "Content: Nid" field added at the top of the fields list, so that it groups by that first. Might need to apply "MIN' to the title (by setting: Aggregation type to "Minimum" in "Aggregation settings" for that field).
Note that once you do that, the image will stop displaying. The solution for that is to go into "Aggregation settings" for the image field, and change the "Group column" from "fid" to "Entity ID". That makes it display again.