Feeds - Background process vs Batch API

Question

I've being using Feeds to import a medium sized amount of nodes in a daily basis. I'm using Elysia cron in order to get fine tuning cron settings.

Here's my flow :

1. Every day at 02 am a cron task is triggered
2. This task loads a list of files from its settings, and for each of these files, it triggers a feeds import by calling :

$myFeed->startImport()

If you take a look at this startImport method, you will see something like this :

public function startImport() {
    module_invoke_all('feeds_before_import', $this);
    $config = $this->importer->getConfig();
    if ($config['process_in_background']) {
      $this->startBackgroundJob('import');
    }
    else {
      $this->startBatchAPIJob(t('Importing'), 'import');
    }
}

So, we have two ways of calling the Feeds importer, either using the Background JOB api (which I suppose to be this : https://drupal.org/project/background_process) or the Batch API (https://drupal.org/node/180528).

Since I'm starting the feeds importer from a cron task, which one is more appropriate for this scenario ?

Can we see the Background process as an enhancement of the Batch API ?

Answer 1

I've been digging through this code, and Feeds does not use Background Process.

It uses Job Scheduler

From FeedsSource.inc:

  /**
   * Background job helper. Starts a background job using Job Scheduler.
   *
   * Execute the first batch chunk of a background job on the current page load,
   * moves the rest of the job processing to a cron powered background job.
   *
   * Executing the first batch chunk is important, otherwise, when a user
   * submits a source for import or clearing, we will leave her without any
   * visual indicators of an ongoing job.
   *
   * @see FeedsSource::startImport().
   * @see FeedsSource::startClear().
   *
   * @param $method
   *   Method to execute on importer; one of 'import' or 'clear'.
   *
   * @throws Exception $e
   */
  protected function startBackgroundJob($method) {
    if (FEEDS_BATCH_COMPLETE != $this->$method()) {
      $job = array(
        'type' => $this->id,
        'id' => $this->feed_nid,
        'period' => 0,
        'periodic' => FALSE,
      );
      JobScheduler::get("feeds_source_{$method}")->set($job);
    }
  }

Job Scheduler in turn uses cron to complete jobs, so you're not really saving yourself anything if you are already calling this from cron.

The core Drupal Batch API uses AJAX to break up jobs into batches. You can't start a Batch API batch from cron. If you don't have JS and try to start a Batch with the core API, you'll still need to watch the progress bar move across the screen. If you abandon your browser during a batch, it will stop working.

Contrary to what the previously accepted answer above states, the core Batch API does not use cron

Background process is now a suite of modules, and it does now include a Batch API module. These are 2 separate modules, which are not related to the Job Schedule module. We can't refer to Background Job/Process as 1 thing. These are 2 unrelated modules.

The key differences are as follows:

Drupal Batch API uses your browser to make new requests to the server at regular intervals. This does not work from a cron job

Job Scheduler uses cron to call on a set of scheduled jobs during cron execution. There is a limit on how long the process can run during cron, so this is not ideal for long-running processes. It also is not good if you need your import to occur in a timely manner, when you have your cron job only running at night. Scheduling a new Job during cron is a bit backwards, since it might not be until the next cron run that the job is actually dispatched (depending on if your hook_cron() is called before or after job_scheduler_cron())

Background process / Background Batch uses server-side http requests to chain steps of a process together. This is ideal for long-running processes that need to be started immediately. You can also start a long-running process from hook_cron() with this module, and it will run until finished.

I know you probably don't need this information now, but I did when I found this page from Google.

The information provided in the previously accepted answer is incorrect, and led me down the wrong path.

Answer 2

My below response is incorrect, please refer to the accepted answer for an accurate outline. I have left it there to show my thought process, where I got my information from and hopefully to stop others going wrong in the future.

Kyle

---

My understanding is that the Background Job/Process is lightweight and can be easily run at peak times, as it will timeout if it runs for too long (predetermined), and interestingly is not actually a true background process module. However, the Batch API simply runs the jobs from the Cron as quickly as possible. I think it may be a case of semantics here, and that they both the the same job, however Background Job may be a little more convoluted in how it operates. Personally, at 0200 there is next to no activity on my site, so I would run my imports through Batch. If, however, you find you are under high stress (or have a low capacity server) at this time, then I would suggest Background Process/Jobs.

Hopefully this helps!