Edit the Body Field Rewrite results in your Views and enable the option Strip HTML tags. Use Preserve certain tags if you have other elements within your text you want to show, enter them there.
This option is below the Trim this field to a maximum number of characters you're probably using, you'll probably want to also use the suboption Field can contain ...
It seems this issue is being worked on at : #2568289: Not possible to reuse field formatters between entity_reference, file, and image fields
For now I've used the following code - included in the mytheme.theme file. (view mode custom_generic_file)
* Implements hook_preprocess_field().