Ok, so this is a weird one...
I have a HEADless Drupal that I'm using to serve webform submissions. The webform module stores all files as private files.
The issues I'm having is if I view the submission in the angular app as the user that submitted the webform (owner) all the files come back fine. If I try to view the submission as a different user the files come back 403 forbidden. All of the other REST data comes back fine.
Then if I go back and view the submission as the owner again the files are now 403 forbidden for the owner as well. The only way I can get the owner to get a 200 response and see the files again is to flush the cache.
Other users viewing those files seem to lock the files down for the owner as well.
Viewing the webform submission in the actual Drupal has no issues.