0
  1. I have a custom entity type and use Feeds to populate the entities. All works great on initial import.
  2. I then run a cron job that updates some fields of the entity that are not imported from the feed by accessing an external API. Again, works great.

Finally, I try to reimport the feed and any entity that was updated in step 2 now produces the error

The rovi game Metroid Prime 2: Echoes (21032) failed to validate with the following errors:
: The content has either been modified by another user, or you have already submitted modifications. As a result, your changes cannot be saved.
Please check your mappings.

The processor settings are:

Insert new rovi games
[*] Insert new rovi games
[ ] Do not insert new rovi games

Update existing rovi games
[ ] Do not update existing rovi games
[ ] Replace existing rovi games
[*] Update existing rovi games

Previously imported items
Unpublish

Expire rovi games
Never

Advanced settings
[✓]New Revision

Is there any way to get feeds to reimport even when the entity has been modified?

Here is the (sanitized) code that the cron job is running.

/**
 * Implements hook_cron().
 */
function msul_rovi_games_cron(){

  define('ITEMS_PER_CRON',175);
  define('STALE_ITEM_UPDATE_TIME',strtotime("-30 days"));

  $nextOffset = \Drupal::state()->get('msul_rovi_games.last_game_id',0);
  $count = {{get total number of items}};
  if($nextOffset>$count){
    $nextOffset = 0;
  }

  $gameIDsToProcessQuery = \Drupal::entityQuery('rovi_game');

  $gameIDsToProcessQuery
    ->accessCheck(FALSE)
    ->condition('changed',STALE_ITEM_UPDATE_TIME,'<=')
    ->range($nextOffset,ITEMS_PER_CRON);
  $gameIDsToProcess = $gameIDsToProcessQuery->execute();
  $gamesToProcess = \Drupal::entityTypeManager()
      ->getStorage('rovi_game')
      ->loadMultiple($gameIDsToProcess);

  $client = new Client();

  foreach($gamesToProcess as $key=>$game){

    // compute catalog query
    $callNumber = {{Compute Call Number from other info}};

    if($game->changed->getValue()[0]['value'] < STALE_ITEM_UPDATE_TIME){
      try {
        $response = {{some api call}};
        $records = json_decode($response->getBody(),TRUE);
      }
      catch(RequestException $e){
        //error handling
      }
      
      $record = $records['records'][0];
      // if found in catalog, update to match catalog
      $changed=FALSE;
      if(!isset($game->cataloged->getValue()[0]['value']) 
          || $game->cataloged->getValue()[0]['value'] != TRUE){
        $game->set('cataloged',TRUE);
        $changed=TRUE;
      }
      //similar if clause repeated for other fields

      if($changed){
        $game->set('changed',time());
        $game->save();
      }
    }
  }

  \Drupal::state()->set('msul_rovi_games.last_game_id',
                        $nextOffset + ITEMS_PER_CRON);
}

1 Answer 1

0

Normally, the Feeds module will allow you to re-import feeds even if the entities have been modified.

For example, on a testing server, I import users from a feed, and my tests make some changes to the user field data. Then I re-import the user feed and the users are reset to the original state.

So, there must be something wrong in your specific case. My guess is that the code that runs in the cron job is not saving the entities properly.

Instead of a three-step approach (import feed, use cron to query external API, re-import feed), I would do everything in one step.

There is a great module called Feeds Tamper. This module allows you to "tamper" the values that are to be imported by feeds. You can also create your own plugin, and it is likely that you can do your calls to the external API here.

If you need to use cron because you will make too many API calls to the external API, you can still use this approach-- just import your feed in the background (what Feeds calls cron import).

2
  • While I appreciate the suggestion, the items pulled from the API are pulled multiple times between each import, so making it part of the feed is not practical. Your suggestion that the cron job is not saving the entities properly is intriguing. Can you point me to more info? I'll modify my original question to indicate the code that the cron job is running. Commented May 17 at 12:39
  • @KevinFinkenbinder You can use Xdebug to step through the code. I would set breakpoints on the custom cron hook and the entity's preSave() and save() methods. Before that, though, I would check the Feeds mappings and make sure unique is set correctly. For example, repeatedly importing the Feed should result in a message that there is nothing new to import. Commented May 17 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.