From what I understand you have a distributor node that has a term reference (multi value) to Product categories.
In your view that lists the distributors you must add a relationship: Content: Taxonomy terms on node. When presented with a choice of vocabularies you select 'Product Category'.
Then you add two fields: Taxonomy term: Name, and Taxonomy term: Image (your image field might be named slightly differently). For both of these fields, during the multi form process of adding them, you will be presented with a form that says for instance: Configure field: Taxonomy term: Name. Here you must select the Relationship as term.
This will create one row for each taxonomy term on the node, which might not be what you want. You can now group it under the content:title of the distributor.
There are two other ways that I can see by which you can group all of the images from one distributor under a single distributor.
First method:
Create a new field: global custom text. Make sure that it is below all of the distributor fields. Move the image field below it. In this global text field add all of the distributor fields (using html and replacement patterns) except for the image field. For instance, you might have something like this:
<div>[title]</div>
<div>[address]</div>
<div>[phone]</div>
Now go and hide all of the fields, the global text field as well, except for the image field. Then in your grouping, select this new global text field as the field to group on. Now apply some CSS to make it look good. And if you want to get rid of that pesky h3 tag around the grouping header, then override the theme view template. Click on Theme: Information in views and select one of the Style output templates.
Second method:
Install the Views Field View module. This allows you to create a field in a view that will be the result of another view.
Create a new view that takes as contextual filter the nid of the distributor, and then as before add a relationship to the terms etc. so that this view only outputs the pictures in groups.
In your original view remove the relationship to the terms, and insert a field to the new view, and make sure to pass it the parameter of the nid.