Handling custom parsing and importing with Feeds

Question

I've been wading through the Feeds module and documentation, and for whatever reason have been unable to figure the best way to do this, so I'm hoping someone will be able to point me in the right direction.

I need to get video node information from one D6 site to another. I've set up a feed via Views on the source site, and I want to use Feeds to ingest the info into the target site. There are two custom things I need to do, and I can't figure the best way to do them.

1. There are two taxonomy vocabularies on the source site that I need to get to specific vocabularies on the target site. One is a tags vocabulary, and one is titled Channel (same on both sites). I've added the tid for the target Channel term to the feed as <channel-map> and I need to import that value into the assigned Channel term on the target node. The problem is that the core Taxonomy module dumps all taxonomy terms, regardless of vocabulary, into the source feed as <category>; the only way to differentiate between vocabs when importing the feed item is to look at the domain attribute, which is the URL for that term. There is a Tags target in the standard feed importer, but I need to filter out the Channel terms, as well as the terms for other vocabs that only exist on the source site.

2. The target site has a custom video formatter that plays the embedded video from the source site when you enter the source site URL into the Video field on the target node in the format http://mysourcesite.com/node/$nid. I've added the source nid to the feed as <source-nid>, so I just need to take that value and construct the URL the source site URL is already stored in a variable on the target site as part of the custom formatter settings), and then insert it into the Video field on the new target node.

So what I'm trying to figure out is the best way to handle these instances when importing the feed. It looks like one option is to write a custom parser and/or importer plugin. However, I don't need to rewrite the whole parsing and importing processes; I just need to handle the two cases above. There are already targets defined for the two fields I need to import into, so would it just be a matter of using the feeds_node_processor_targets_alter() hook and override the existing node processor? Or would I need to write my own custom plugins?

Thanks.

EDIT: I tried creating my own callback as a wrapper for content_taxonomy_feeds_set_target() using hook_feeds_node_processor_targets_alter(), but by that point in the process, all that is provided is the term name with no additional information to indicate the source vocabulary.

Answer 1

I finally got everything to work with some code voodoo and changing parsers. One thing that I didn't add to the original question above as an edit was the fact that using the common syndication parser, my custom items that I added to the source feed were never available. I then switched to using the SimplePie parser, and was able to extend that functionality to do what I need.

According to the Feeds module documentation for developers and this blog post, you can do some custom parsing by overriding the parseExtensions() method of FeedsSimplePieParser class. First, you have to declare the class with hook_feeds_plugins():

/**
 * Implementation of hook_feeds_plugins().
 */
function mymodule_feeds_plugins() {
  $info = array();

  $info['MyModuleFeedsParser'] = array(
    'name' => 'Custom Parser',
    'description' => 'Custom parser for my video import.',
    'handler' => array(
      'parent' => 'FeedsSimplePieParser',
      'class' => 'MyModuleFeedsParser',
      'file' => 'MyModuleFeedsParser.inc',
      'path' => drupal_get_path('module', 'mymodule') . '/plugins/feeds/',
    ),
  );
  return $info;
}

You also want to implement hook_enable to clear caches.

/**
 * Implementation of hook_enable.
 */
function mymodule_enable() {
  //clear the cache to display in Feeds as available plugin.
  cache_clear_all('plugins:feeds:plugins', 'cache');
}

Next you create the class.

/**
 * Custom Feeds parser class used to parse custom info from source site video feed.
 */

class MyModuleFeedsParser extends FeedsSimplePieParser {
  /**
   * Add the extra mapping sources provided by this parser.
   */
  public function getMappingSources() {
    return parent::getMappingSources() + array(
      'source-nid' => array(
        'name' => t('Video NID - SimplePie'),
        'description' => t('.'),
      ),
      'channel-map' => array(
        'name' => t('Channel Map - SimplePie'),
        'description' => t('Target Channel tid - SimplePie.'),
      ),
    );
  }

  /**
   * Parse the extra mapping sources provided by this parser.
   */
  protected function parseExtensions(&$item, $simplepie_item) {
    if ($value = $simplepie_item->get_item_tags(SIMPLEPIE_NAMESPACE_RSS_20, 'source-nid')) {
      $item['source-nid'] = $value[0]['data'];
    }
    if ($value = $simplepie_item->get_item_tags(SIMPLEPIE_NAMESPACE_RSS_20, 'channel-map')) {
      $item['channel-map'] = $value[0]['data'];
    }
    // Filter out non-Tags terms.
    $categories = $simplepie_item->get_categories();
    // Cycle through the categories and delete any items
    // in feed item that aren't tags. The array id values
    // are synchronized between $categories and $item['tags'].
    foreach($categories as $id => $category) {
      if (!strstr($category->scheme, '/tag/')) {
        unset($item['tags'][$id]);
      }
    }
  }
}

This code does a few things:

1. getMappingSources() creates a couple sources that are available for mapping. You set these in your importer settings for your processor.
2. Gets the source-nid value from the feed source and add it to the item.
3. Gets the channel-map value from the feed source and add it to the item.
4. Filters out the non-tags terms from the categories (aka taxonomy terms). The scheme value is the URL to the taxonomy term (e.g. mysite.com/tag/tag-term), so I use that to identify non-tag terms and unset them.

Finally, I have to create the values for the video field and the Channel vocabulary. This is done with some custom callbacks defined in hook_feeds_node_processor_targets_alter().

/**
 * Implements hook_feeds_node_processor_targets_alter().
 */
function mymodule_feeds_node_processor_targets_alter(&$targets, $entity_type, $bundle_name) {
  $targets['field_source_url'] = array(
    'name' => 'Source Video URL',
    'description' => 'The URL of the source video',
    'callback' => 'mymodule_feeds_set_source_url',
  );

  $targets['channel'] = array(
    'name' => 'Target Channel',
    'description' => 'The target video channel',
    'callback' => 'mymodule_feeds_set_channel_target',
  );
}

And the callbacks themselves:

/**
 * Custom callback to set the video source URL in field_video.
 *
 * @param $node - The node object being built.
 * @param $target - The target field.
 * @param $value - The field value from the feed.
 */
function mymodule_feeds_set_source_url(&$node, $target, $value) {
  $source_url = variable_get('emvideo_source_domain', '') . '/node/' . $value;
  $node->field_video[0]['embed'] = $source_url;
}

/**
 * Custom callback to set the target vocabulary term.
 *
 * @param $node - The node object being built.
 * @param $target - The target field.
 * @param $value - The field value from the feed.
 */
function mymodule_feeds_set_channel_target(&$node, $target, $value) {
  $term = taxonomy_get_term($value);
  $node->taxonomy[(int)$value] = $term;
}

You then define these as targets in your Processor mapping (to coincide with the Source values above), and you're good to go.