Is it possible to convert separate (matching) managed files to a single shared file?

Question

I'm finishing up a Commerce store that has tens of thousands of imported products. In a slightly simplified form, here's my problem: Each has a unique primary image and a common secondary image. The import has created a separate secondary image of the exact same file for each product, which obviously creates enormous redundancy and uses much disk space unnecessarily.

Is there a safe way to convert all of the secondary images into a single, shared image in the database and filesystem? (I'm envisioning some kind of database SQL to make multiple products reference the same image, and then hoping that Drupal will clean up the orphan files automatically -- am I looking in the right direction?)

(Drupal 7)

Answer 1

For small scale issues of this nature, the Duplicate Images module may be exactly what you need to step through the process of identifying and removing duplicates. (At the moment it bogs down and fails with huge numbers of duplicate images).

What does this module do?

In a multi step form (like update.php) this module executes the following steps:

1) Search for duplicates

• Searches are done on the public and/or private file system
• It searches for file names ('filename' as used by pathinfo) that end with _{n} and whose part before that also exists as separate file name.
• It compares file size and md5 hash (see md5_file) to determine if these are real duplicates or possible duplicates.
• The results are presented as a list whereby for the possible duplicates clickable thumbnails are shown so you can visually compare them. For documents just a clickable icon is shown.

2) Search for usages

• Look if a managed file record is defined for the duplicate (and the original).
• Searches for references to the managed file in user pictures, all image and file fields, all fields that according to their field schema have a foreign key to the file_managed table.
• Searches for URI references to the file or a(n image style) derivative in selected text and link fields.

3) Update usages

• Found references to the managed file record are updated to refer to the managed file record of the original (or if that does not yet exist, the uri field is simply updated).
• Found textual usages are changed to refer to the URI of the original document.
• Note 1: this phase uses the entity_save() function of the entity api (contrib and thus a dependency) to ensure that caches are cleared, hooks are called, file_usage is update, rules are executed, etc.
• Note 2: this phase does keep track of failed updates so that the next phase can skip managed file records or files that are still being referred to.

4) Delete duplicates

• All managed file records that are no longer referred to are deleted.
• All duplicate files that ar no longer referred to are deleted.