How do I create and synchronize product fields with entity fields on another site

Question

Case description:

I have one centralized 'Asset management' Drupal website "A" (something like a simple ERP site) which manages the administration of purchased Assets, being ECK entities. The assets are unique, no stock management required.

I also have a Drupal webshop "B" using the Drupal Commerce modules, which has Commerce Product entities.

Goal:

The assets and products need to be two-way synchronized between the two sites, in such a way that:

• New Assets in site A will create new Products in site B.
• Some fields (title, price, status) will need to be two-way syncronized (so that when a product is sold, the Asset status field will be changed)

Requirements

Preferably without custom coding, but using existing Drupal modules in combination with Rules to allow flexibility in configuration.

Question:

• What would be a good approach in this case? Preferably without custom coding.

My current investigations:

There are multiple ways to construct a non-coding contributed module combination to get the result needed. My preference is a lightweight but configurable solution using Rules. I checked out these modules:

• Services module: Without other modules this requires custom coding to import and sync entities. Most services-based modules are dependent on it.
• Web Service Data module, This module allows you to bypass the default data storage location for fields and entities and instead load the data from a web service. Does not import or sync. Probably not suitable because the entities are not identical (assets and products)
• Remote Entity Api module but this might not be the most efficient way because that module creates local copies of the remote entities. This means that in site B there would exist also the same amount of asset entities as in site A, which are synchronized with the Product entities.
• Feeds module allows one-way import and sync, could be two-way but probably complex.
• The Migrate module: large framework, is a developer's tool, must have php skills.
• Services Client module the UUID needs to be inserted per entity? Not for many entities.
• Service Rules module creates endpoint for Rules Components. Requires a services client module. Might be of use (optional, see below)

[UPDATE MAY 30 2016]

According to my research the following modules combined, will likely be the most lightweight and configurable solution. I take into account that this case is not so much about migration of data, but about creating and updating using rules triggers and actions (as per the requirements).

• Web Service Client module is made to consume services without custom coding. It integrates with Rules to perform the required actions.
• Web Service Client for Feeds integrates the Web Servcice Client module with Feeds, which allows for a cron-based import of created or changed entities.
• Sevices Views module: in order to create a view of the assets fields that need to be created or updated.

TO DO: The suggested approach mentioned above needs to be tested in order to prove that it works. I still don't quite understand how to configure the Web Service Client module configuration of the 'REST service descriptions'. Probably because I am a newbie to services. In order to figure that out, I posted a question about it. I will add screenshots to that where needed.

For those answering this BONUS question: The answers have been rather broad up to now. Using the abovementioned update (may 30) please provide specific support for the solution suggested by me using the Web Service Client module configuration. This allows me to choose who gets the bonus. You may answer in the connected question in order to keep the answers on topic per question. I have also created two test sites with the modules installed and can provide admin login details if you want to test it out, or help me out with the configuration.

Answer 1

I'll start with saying, you should throw your "no custom coding" requirement out. That's not to say it's not possible, but it's going to require you stitching a patchwork of solutions that might drive you to conflicting or cumbersome processes. Every data migration problem, save for the most basic (e.g. migrating content title & body nodes), will have key details and rarely has pre-made default path in Drupal.

You don't want a patchwork solution to a detailed problem like this; it will be difficult to manage if you find your future requirements or key module assumptions change in the future. Rather, you should rely on a few key tools to handle some big needs, and fill in the rest with custom code that leverages either Drupal's or contrib module's APIs.

You have 2 broad requirements. I'll list an outline of one approach.

1. An ETL solution between a new custom asset entities on Site A and commerce product on Site B.
   • You should, from the start, utilize a shared UUID on your asset entity & Commerce product; the alternative here is having some sort of shared PK mapping between sites that could have all sorts of problems.
   • The go-to-solution for Drupal data ETL is the Migration framework that allows you specify how to map fields, transform them if needed, and place them into your migrated destination. The duplicate insert problem is solved with UUIDs. Commerce Migrate provides extensions specific to the Commerce project.
   • You'll need to determine when/how such a migration sync will occur. A solution will need to be driven by your own workflow. As such, there's probably no one-size-fits-all solution here (e.g. if assets are periodically added on scheduled date a simple cron job to migrate products in might suffice a real-time requirement might require some RPC solution).
2. Data Field syncing.
   • You should rely on Serivces (as you have mentioned) and Services Entity API to provide CRUD transport. This will allow reliable data push/pull when you need it. Additionally, it could be the framework for providing a RPC service endpoint in the event you want to trigger an asset-to-commerce migration on site B from site A.
   • Just like with asset-to-product ETL, your sites' workflow is probably going to determine the best course of action. It could work with Rules (with a module provide custom conditions & actions to determine when to sync up via REST) or perhaps a simple custom module running a cron job.

In short, don't rely on a patchwork of module projects to somehow solve the problem for you through trial-and-error. Pick key tools that solve one single problem well (e.g. ETL, HTTP data transport, unique identification) and use your custom code to supply the logic that's unique to your use case.

Answer 2

Cross-site sync in Drupal 7 is non-trivial (there's a large project happening to make it easy in Drupal 8...some day), and since everything can have customizations general purpose tools have to be so flexible as to require massive effort to develop. True, there is lots of desire but rarely do two people have a shared set of requirements.

I've built a couple syncing solutions and usually at their heart they become CSV or JSON file exchanges. So while I've always used either custom code or a human to solve the hard problems to avoid duplicates and update loops, it might be possible to wire together feeds (using a JSON parser) and views (with a JSON display) or services in a way that would push information back and forth as long as it stays as simple as you suggested in your original post.