Add column to custom field schema

Question

I have created a D8 custom field that extends FieldBaseItem. It is already in use storing data.

How do I get the schema updated in the database, for an existing field?

What I've done:

I have added a new column by adding to propertyDefinitions and schema() accordingly.

Here's an example schema. Let's say I just added the admin column.

  /**
   * {@inheritdoc}
   */
  public static function schema(FieldStorageDefinitionInterface $field_definition) {
    $schema = [
      'columns' => [
        'state' => [
          'type' => 'varchar',
          'length' => 2
        ],
        'admin' => [
          'type' => 'int',
          'size' => 'tiny'
        ],
        ...

I've found posts on how to do this in hook_update_N in D7, but it's unclear how to do this in D8.

Answer 1

In fact, like it's a custom field type, your change in the schema can be detected and applied if you launch the drush command drush entity-updates (or drush entup).

All tables which implement this field (like node__field_my_field_type or paragraph__field_my_field_type) will be updated. So you don't need to use an hook_update to update the schema of all these tables.

The big problem with this command is many errors are silent and sometimes, the command don't return errors but the schema hasn't been updated and still proposed by the command to be updated.

Answer 2

Since entup is no longer available in Drupal 8 (or 9 or 10), many people have struggled with this, and many modules contain copious amounts of code in their update hooks to deal with this. If you know exactly which field instances need to be updated, you can update their tables directly, but this is not an option if you have a module that provides a FieldType, and you are writing an update hook for that module. You want all instances of your field type to be detected and updated.

This Drupal core issue has some code examples for adding and removing properties from existing field types while updating all instances of that field type: https://www.drupal.org/project/drupal/issues/937442#comment-13760432 . Credit goes to https://www.drupal.org/u/robbinzhao . I just cleaned up the code and made it into a utility class:

<?php

namespace Drupal\my_utilities;

use Drupal\Core\Entity\Sql\SqlContentEntityStorage;

/**
 * Utilities for updating field type definitions.
 *
 * Based on https://www.drupal.org/project/drupal/issues/937442#comment-12586376
 */
class FieldTypeUpdateUtil {

  /**
   * Add a new column for fieldType.
   *
   * @param string $field_type
   *   The ID of the field type definition.
   * @param string $property
   *   The name of the property whose column is to add.
   *
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   * @throws \Drupal\Component\Plugin\Exception\PluginNotFoundException
   * @throws \Drupal\Core\Database\SchemaObjectDoesNotExistException
   * @throws \Drupal\Core\Database\SchemaObjectExistsException
   */
  public static function addProperty($field_type, $property) {
    $database_schema = \Drupal::database()->schema();

    $entity_definition_update_manager = \Drupal::entityDefinitionUpdateManager();

    // First, get all fields of the provided field type.
    $field_map = \Drupal::service('entity_field.manager')->getFieldMapByFieldType($field_type);

    // Process per entity type (i.e., node, paragraph) all fields.
    foreach ($field_map as $entity_type_id => $fields) {
      $entity_type_storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);

      // Database schema can only be updated, if the values are stored in the
      // database, of course.
      if ($entity_type_storage instanceof SqlContentEntityStorage) {
        foreach (array_keys($fields) as $field_name) {
          $field_storage_definition = $entity_definition_update_manager->getFieldStorageDefinition($field_name, $entity_type_id);

          // Database column definition, according to each field property.
          $field_columns = $field_storage_definition->getColumns();

          // Get the table and column names, according to the field definition.
          $table_mapping = $entity_type_storage->getTableMapping([
            $field_name => $field_storage_definition,
          ]);
          $table_names = $table_mapping->getDedicatedTableNames();
          $table_columns = $table_mapping->getColumnNames($field_name);

          // Check if, for the property, the corresponding table exists in the
          // database. If not, create the table column (according to the schema
          // provided by the field definition).
          foreach ($table_names as $table_name) {
            $table_exists = $database_schema->tableExists($table_name);
            $field_exists = $database_schema->fieldExists($table_name, $table_columns[$property]);

            if ($table_exists && !$field_exists) {
              $database_schema->addField($table_name, $table_columns[$property], $field_columns[$property]);
            }
          }

          // Reflect any changes made to the field storage definition.
          $entity_definition_update_manager->updateFieldStorageDefinition($field_storage_definition);
        }
      }
    }
  }

  /**
   * Remove a property and column from field_type.
   *
   * @param string $field_type
   *   The ID of the field type definition.
   * @param string $property
   *   The name of the property whose column to remove.
   *
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   * @throws \Drupal\Component\Plugin\Exception\PluginNotFoundException
   * @throws \Drupal\Core\Entity\Sql\SqlContentEntityStorageException
   */
  public static function removeProperty($field_type, $property) {
    $field_map = \Drupal::service('entity_field.manager')
      ->getFieldMapByFieldType($field_type);
    foreach ($field_map as $entity_type_id => $fields) {
      foreach (array_keys($fields) as $field_name) {
        self::removePropertyFromEntityType($entity_type_id, $field_name, $property);
      }
    }
  }

  /**
   * Inner function, called by removeProperty.
   *
   * @param string $entity_type_id
   *   The ID of the entity type.
   * @param string $field_name
   *   The ID of the field type definition.
   * @param string $property
   *   The name of the property whose column to remove.
   *
   * @throws \Drupal\Component\Plugin\Exception\PluginNotFoundException
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   * @throws \Drupal\Core\Entity\Sql\SqlContentEntityStorageException
   */
  private static function removePropertyFromEntityType($entity_type_id, $field_name, $property) {
    $entity_type_manager = \Drupal::entityTypeManager();
    $entity_update_manager = \Drupal::entityDefinitionUpdateManager();
    $entity_storage_schema_sql = \Drupal::keyValue('entity.storage_schema.sql');

    $entity_type = $entity_type_manager->getDefinition($entity_type_id);
    $field_storage_definition = $entity_update_manager->getFieldStorageDefinition($field_name, $entity_type_id);
    $entity_storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);
    /** @var \Drupal\Core\Entity\Sql\DefaultTableMapping $table_mapping */
    $table_mapping = $entity_storage->getTableMapping([
      $field_name => $field_storage_definition,
    ]);

    // Load the installed field schema so that it can be updated.
    $schema_key = "$entity_type_id.field_schema_data.$field_name";
    $field_schema_data = $entity_storage_schema_sql->get($schema_key);

    // Get table name and revision table name, getFieldTableName NOT WORK,
    // so use getDedicatedDataTableName.
    $table = $table_mapping->getDedicatedDataTableName($field_storage_definition);
    // try/catch.
    $revision_table = NULL;
    if ($entity_type->isRevisionable() && $field_storage_definition->isRevisionable()) {
      if ($table_mapping->requiresDedicatedTableStorage($field_storage_definition)) {
        $revision_table = $table_mapping->getDedicatedRevisionTableName($field_storage_definition);
      }
      elseif ($table_mapping->allowsSharedTableStorage($field_storage_definition)) {
        $revision_table = $entity_type->getRevisionDataTable() ?: $entity_type->getRevisionTable();
      }
    }

    // Save changes to the installed field schema.
    if ($field_schema_data) {
      $_column = $table_mapping->getFieldColumnName($field_storage_definition, $property);
      // Update schema definition in the database.
      unset($field_schema_data[$table]['fields'][$_column]);
      if ($revision_table) {
        unset($field_schema_data[$revision_table]['fields'][$_column]);
      }
      $entity_storage_schema_sql->set($schema_key, $field_schema_data);
      // Try to drop field data.
      \Drupal::database()->schema()->dropField($table, $_column);
    }
  }

}

Answer 3

The update hooks are essentially the same as Drupal 7, but it's recommended to use the DB Connection object with the addField() method.

The update hook to add a column might look like this:

$spec = array(
 'type' => 'varchar',
 'description' => "New Col",
 'length' => 20,
 'not null' => FALSE,
); 
$schema = Database::getConnection()->schema();
$schema->addField('mytable1', 'newcol', $spec);

Full documentation can be found here: https://www.drupal.org/docs/8/api/update-api/updating-database-schema-andor-data-in-drupal-8

Answer 4

This is a function I wrote to add new field(s) to existing field tables.

/**
 * Add a new column for fieldType.
 *
 * @param string   $field_type  ID of the field type.
 * @param string[] $new_fields  Field key(s) that is new to your schema.
 *
 * @return void
 */
function field_type_add_property(string $field_type, array $new_fields) {

  if (!$field_storage_configs = \Drupal::entityTypeManager()
    ->getStorage('field_storage_config')
    ->loadByProperties(array('type' => $field_type))
  ) {
    throw new \Exception('Unable to find field type ' . $field_type);
  }

  // Get schema handler to alter table with.
  $schema_handler = Database::getConnection()->schema();

  // Loop through all configs.
  foreach ($field_storage_configs as $field_storage) {

    // Just to play safe
    if (!($field_storage instanceof FieldStorageDefinitionInterface)) {
      throw new \Exception('Unexpected entity type loaded: ' .
        (is_object($field_storage) ? get_class($field_storage) : var_export($field_storage, true)));
    }


    // get the most updated schema from field stroage
    $schema = $field_storage->getSchema();
    $table_name = str_replace('.', '__', $field_storage->id());
    $revision_table_name = str_replace('.', '_revision__', $field_storage->id());

    // loop through the fields to alter the schema of table.
    foreach ($new_fields as $new_field) {
      $column_name = "{$field_storage->getName()}_{$new_field}";
      if (!$schema_handler->fieldExists($table_name, $column_name)) {
        $schema_handler->addField(
          $table_name,
          $column_name,
          $schema['columns'][$new_field]
        );
      } else {
        // Show some warning message or error
        throw new \Exception(sprintf('Field %s already exists in table %s',
          $column_name, $table_name));
      }
      if (!$schema_handler->fieldExists($revision_table_name, $column_name)) {
        $schema_handler->addField(
          $revision_table_name,
          $column_name,
          $schema['columns'][$new_field]
        );
      } else {
        // Show some warning message or error
        throw new \Exception(sprintf('Field %s already exists in table %s',
          $column_name, $revision_table_name));
      }
    }
  }
}