For the tagging part, forget about entity browser, that (currently) doesn't know about the context of where it is invoked and to what those files will eventually be added.

Instead, what you want to hook into is hook_node_presave(). Go through the referenced files in all your fields that you want to support (could be all image fields or a configurable list), check the image referenced there. If the image doesn't have a category yet, set it based on the field that you're currently looking. Or maybe always ensure it is categorized based on the current field. Whatever you want.

For views, that's currently not possible yet, for the same reason. Entity browser doesn't receive the context yet. There are some related issues like https://www.drupal.org/node/2656196 and https://www.drupal.org/node/2366335. Note that there are many ways to embed an entity browser and you don't always have a field, for example  with entity_embed.