Start next batch process automatically

Question

Is it possible to start next batch_set() after finishing first one? I've some import functionality which includes 4 steps. The next step takes data processed from first batch process so this is consecutive process which needs to be run one after one.

Answer 1

I have done this before using Core drupal tools. Although I suspect different than your current approach.

I use a drupal Queue. and I fill it with items:

                                                    Queue
          Task1Items              Task2Items          Task3Items              Task4Items
    [---item1---item2---item3|item4---item5---item6|item7---item8---item9|item10---item11---item12]

I put all the tasks, in the ordering for the various steps of your task into a Queue. Then be sure to use a Drupal Reliable Queue:

There are two kinds of (default) queue backends available: reliable, which preserves the order of messages and guarantees that every item will be executed at least once. The non-reliable kind only does a best effort to preserve order in messages and to execute them at least once but there is a small chance that some items get lost. For example, some distributed back-ends like Amazon SQS will be managing jobs for a large set of producers and consumers where a strict FIFO ordering will likely not be preserved. Another example would be an in-memory queue backend which might lose items if it crashes.

After inserting all the items. Your batch call simply iterates over the queue you've populated. Use something like Ultimate Cron to setup a group of task workers, or Background Process, or Ultimate Cron Queue Scalar to speedup your workload if you want.

Another handy module to view your queue visually is the Queue UI module. As the core Queue API is a developer API and has no UI by itself. Don't want to write php code to create/manage a queue -- look at Rules Queue.