You need to add entity_type_id aswell in the hook.
It should be
use Drupal\Core\Entity\EntityTypeInterface;
/**
* Implements hook_entity_bundle_field_info_alter().
*/
function mymodule_entity_bundle_field_info_alter(&$fields, EntityTypeInterface $entity_type, $bundle)
{
if ($entity_type->id() === 'paragraph' && $bundle === 'my_paragraph_bundle' && isset($fields['my_paragraph_field'])) {
$fields['my_paragraph_field']->addConstraint('MyConstraint', []);
}
}
For reference : Adding Constraint to entity referenced Paragraph field