It appears as though the issue I had relates to this issue: Display mode configurations don't get updated with new fields https://www.drupal.org/project/drupal/issues/2915036 I haven't tried the patch yet, but after saving the individual display modes the fields then get put into the export and sync properly.

Only top voted, non community-wiki answers of a minimum length are eligible