So this worked for me not sure why. But my file migration id was
upgrade_d6_file and when I change this in line 54 of Drupal\file\Plugin\migrate\process\d6\FieldFile and ran the migration with the settings below I got the correctly referenced files and the alt/title text.
// Configure the migration process plugin to look up migrated IDs from
// a d6 file migration.
$migration_plugin_configuration = $configuration + [
- 'migration' => 'd6_file',
+ 'migration' => 'upgrade_d6_file',
'source' => ['fid'],
Or even better is write your own process plugin like this: https://github.com/mikeryan776/d8-migrate-example-001/blob/master/migrate_carlyle/src/Plugin/migrate/process/CarlyleCckFile.php