I have two JSON source files of product categories which contain duplicate products.
Here's the link of the two source JSON files https://gist.github.com/naveenvalecha/427e32291c0247553bba9586448e4224
I'm traversing the JSON data using the JSONPATH migrate_plus data parser plugin(https://www.drupal.org/node/3007709) with item_selector $.._element[*] which is returning the arrays of all products under a respective category JSON file.
If a product has multiple categories, it is present in both categories JSON files. Migration status shows the wrong count because it is considering the same product twice which is present in both categories JSON file.
Will it be reliable to merge the source records of different JSON sources when there are thousands of records?
Note: There could be multiple JSON product category JSON source files.
Right now, When I import the products. if a product is present in both of the categories. On the first run, it gets migrated. When the same product gets imported again, it replaces the values of the same content with the new values of the source.
Will it be reliable to merge the field_category values when the same record gets updated again as there are thousands of records?
The migration template(cf_product.yml) is also attached in the gist https://gist.github.com/naveenvalecha/427e32291c0247553bba9586448e4224
