0

My feeds importer is acting very strange and running the same import multiple times..

This is what my importer looks like:

class FeedsASUNewCareer extends FeedsCSVParser {
  /**
   * Implements FeedsParser::parse().
   */
  public function parse(FeedsSource $source, FeedsFetcherResult $fetcher_result) {
    global $user;
    $source_config = $source->getConfigFor($this);
    $state = $source->state(FEEDS_PARSE);

    // Load and configure parser.
    feeds_include_library('ParserCSV.inc', 'ParserCSV');
    $parser = new ParserCSV();
    $delimiter = $source_config['delimiter'] == 'TAB' ? "\t" : $source_config['delimiter'];
    $parser->setDelimiter($delimiter);

    $iterator = new ParserCSVIterator($fetcher_result->getFilePath());
    if (empty($source_config['no_headers'])) {
      // Get first line and use it for column names, convert them to lower case.
      $header = $this->parseHeader($parser, $iterator);
      if (!$header) {
        watchdog('CAREER IMPORT ERROR', "Could not parse header");
        return;
      }
      $parser->setColumnNames($header);
    }

    // Determine section to parse, parse.
    $start = $state->pointer ? $state->pointer : $parser->lastLinePos();
    $limit = $source->importer->getLimit();
    $rows = $this->parseItems($parser, $iterator, $start, $limit);

    // Report progress.
    $state->total = filesize($fetcher_result->getFilePath());
    $state->pointer = $parser->lastLinePos();
    $progress = $parser->lastLinePos() ? $parser->lastLinePos() : $state->total;
    $state->progress($state->total, $progress);

    $careers = array();

    watchdog('careers', '<pre>' . print_r($rows, TRUE) . '</pre>');
    foreach($rows as $career) {
        // create career.
        $node = entity_create('node', array('type' => 'careers'));
        $entity = entity_metadata_wrapper('node',$node);
        // Set Entity properties
        $entity->title->set($career['alternate_title']);
        $entity->language = 'en';
        $entity->author = $user->uid;
        $entity->status = $career['status'];
        $entity->body->set(array('value' => $career['body']));
        $entity->field_alternate_title->set($career['alternate_title']);
        $entity->field_career_category_image->set($career['car_category_img']);
        $entity->field_careercategory->set($setCategoryId);

        // Save Career
        $entity->save(); //Save new Entity wrapper
        $node->tnid = $node->nid;
        $entity->save();
      }
    }

    // return empty set.
    $ret = new FeedsParserResult();
    return $ret;
  }
}

When I go to Content > Import and select this Feed Importer to use, it just gets hung up on the processing page and in my logs I can see my watchdog watchdog('careers', '<pre>' . print_r($rows, TRUE) . '</pre>'); executes over and over until I stop it. Keep in mind that watchdog is not in a loop, so the feeds importer is just running endlessly until I stop it.

What could cause the importer to act with such behavior?

0

It could be that there's a bug in the parser's reporting progress. As long as the parser doesn't tell Feeds parsing is done, Feeds will keep calling the parser over and over again.

Workflow of Feeds

  1. Fetch data.
  2. Parse 50 first items of data.
  3. Process 50 items.
  4. Feeds checks if the parser's progress is complete. If not, the next 50 items of data are parsed.
  5. Process 50 items.

When there's nothing more to parse, Feeds checks if there's more to fetch. If so, above steps are repeated. If not, import finishes.

| improve this answer | |
  • So if something in the parser fails, the parser just retries endlessly? That is the worst design I've ever heard of. – BlondeSwan May 18 at 19:08
  • If Feeds hears from the parser that parsing is not complete yet, then Feeds will keep calling the parser until the parser reports it is complete. If the parser fails to report to Feeds that it is complete at some point, then that is a bug in the parser's code. – MegaChriz May 19 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.