I am working on a custom module which processes a JSON file and downloads images from URLs in the JSON object, creates a new file entity, replaces the image URL in the JSON file with the file ID, and again export the JSON file to be downloaded with the Image IDs instead of Image URLs.

I am using Drupal batch to do this as the JSON file contains thousands of records to be processed.

In order to achieve this, as you can see in the code below, I create 2 custom function in .module file and process it in the custom form:

FormFile (sample code)

public function submitForm(array &$form, FormStateInterface $form_state) {
$result_export = $form_state->getValue('result_export');
// Getting the name of directory where the image will be saved in.
$directory = $form_state->getValue('image_dir');
// Getting the main key.
$json_main_key = $form_state->getValue('json_main_key');
// Getting the data from the json_file field.
$json_file_id = $form_state->getValue('json_file');
// Loading the file by file ID.
$json_file =  File::load($json_file_id[0]);
//$json_file = file_load($json_file_id);
// Getting the URL of the file in order to load the data into an array
// as this step will make the data comparision easier than objects.
$json_file = file_get_contents($json_file->getFileUri());
// Defining the array that will hold the data extracted from base_file json
// file.
$json_file_array = [];
// Converting the json data in the base_file into array.
$json_file_array = json_decode($json_file, TRUE);
$nodes = $json_file_array[$json_main_key];
$batch = [
  'title' => t('Download and Import Image'),
  'operations' => [],
  'init_message' => t('Operation in progress'),
  'error_message' => t('The process has encountered an error.'),
  'finished' => '::onFinishBatchCallback',

// creating the parent directory.
// Get a public directory folder.
$public_files = \Drupal::service('file_system')->realpath(file_default_scheme() . "://");
//Check parrent directory and create it if dose not exist.
$parrent_directory = $public_files . '/' . $directory;
file_prepare_directory($parrent_directory, FILE_CREATE_DIRECTORY);
// Start processing the record.
// json_combine_download_image is a custom function to download images,
// create file entites and replace the image URL with the image ID.
foreach ($nodes as $array_key => $row) {
  $batch['operations'][] = ['json_combine_download_image', [$nodes, $row, $array_key, $directory, 'logo', NULL]];
// json_combine_export is a Custom Function to export JSON file.
$batch['operations'][] = ['json_combine_export', [$nodes, $json_main_key, $result_export]];
$context['message'] = 'Operation performed sucessfully';
$context['results'] = $results;



The custom function to process the images is:

function json_combine_download_image($array, $row, $array_key, $directory, $key, $subkey = NULL) {
  if ($row[$key]) {
    if (!is_array($row[$key])) {
      // Preparing the main image to be imported to the field_gallery.
      if (isset($row[$key])) {
        // Get a public directory folder.
        $public_files = \Drupal::service('file_system')->realpath(file_default_scheme() . "://");
        $directory_location = $row['title'];
        // Creating the directory of the scools to wrap each school image.
        $school_path = json_combine_strip_schar($directory_location);
        $school_path = $directory . '/' . $school_path;
        $school_directory = $public_files . '/' . $school_path;
        file_prepare_directory($school_directory, FILE_CREATE_DIRECTORY);
        // Proaparing the path for the image download.
        $image_dir = 'public://' . $school_path;
        $image_value = stripslashes($row[$key]);
        if(json_combine_checkRemoteFile($image_value)) {
          $image = system_retrieve_file($image_value, $image_dir, TRUE, FILE_EXISTS_RENAME);
        //if ($image) {
          $image_id = $image->id();
          $image_id_path = [];
          $image_id_path['path'] = $image_value;
          $image_id_path['id'] = $image_id;
          $array[$array_key][$key] = $image_id;
          //drupal_set_message(t($key . ' Image from ' . $image_value . ' has been imported'), 'status', FALSE);
          drupal_set_message(t('File ID ' . $array[$array_key][$key]), 'status', FALSE);
        else {
          drupal_set_message(t($image_value . ' couldn\'t be downloaded' ), 'error', FALSE);
    return $array;

This $array is the processed one with the I will use as an input for the JSON export custom function which will export the JSON object.

The custom function to export to JSON is:

function json_combine_export($array, $json_main_key, $result_export) {
  $public_files = \Drupal::service('file_system')->realpath(file_default_scheme() . "://");
  $directory = $public_files . '/' . 'export';
  file_prepare_directory($directory, FILE_CREATE_DIRECTORY);
  $json_export = 'public://export/' . $result_export;
   * Exporting the merged JSON file.
  // Encoding the data as JSON.
  $merge_export = [];
  $merge_export[$json_main_key] = $array;
  $json_data = json_encode($merge_export);
  // Saving the the merged data into a file.
  $file = file_save_data($json_data, $json_export, FILE_EXISTS_REPLACE);
  $json_file_id = $file->id();
  $json_file = \Drupal\file\Entity\File::load($json_file_id);
  $json_file_url = file_create_url($json_file->getFileUri());
  $jsone_file_link = '<a href="' . $json_file_url . '"> Download Merged JSON File</a>';
  drupal_set_message(t('<a href="' . $json_file_url . '" target="_blank"> Download Merged JSON File</a>'), 'status', FALSE);

I can't manage to use the $array that is processed from the first function as an input for the second one.

Any ideas?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.