Try a different hook for base fields:

/**
 * Implements hook_entity_base_field_info_alter().
 */
function mymodule_entity_base_field_info_alter(&$fields, EntityTypeInterface $entity_type) {
  if ($entity_type->id() === 'taxonomy_term') {
    $fields['foo']->addConstraint('Length', ['min' => 5]);
  }
}

If the field has bundle specific overrides you might need to use both hooks. See https://www.drupal.org/project/drupal/issues/3193351

Try a different hook for base fields:

/**
 * Implements hook_entity_base_field_info_alter().
 */
function mymodule_entity_base_field_info_alter(&$fields, EntityTypeInterface $entity_type) {
  if ($entity_type->id() === 'taxonomy_term') {
    $fields['foo']->addConstraint('Length', ['min' => 5]);
  }
}

If the field has bundle specific overrides you might need to use both hooks. See https://www.drupal.org/project/drupal/issues/3193351

Example for both hooks found in a contrib module https://git.drupalcode.org/project/node_title_validation/-/blob/8.x-1.x/node_title_validation.module