How to get the uncached version of request headers?

Question

We need to programmatically get the request headers using the mymodule_preprocess_page hook. To do this we tried the following, however, because we have CDN, Varnish, and Drupal Cache all enabled, the request headers are being pulled from cache so if in the initial request the header value we are looking for is set to false, it is false in all requests afterwards.

How can we get the uncached request headers?

Here is what we tried, but it didn't work.

function mymodule_preprocess_page(&$variables) {
  $request = \Drupal::request();
  $request_hdrs = $request->headers;
  // This is the one that seems to be always caching. (true or false) 
  // my-header is coming from another server. 
  $request_custom = $request->get('my-header');

  // log this for troubleshooting
  \Drupal::logger('mymodule')->log(RfcLogLevel::INFRO, 'header value'.$request_custom);

  if($request_custom) {
    // based on the header value, redirect to another url. 
    $response = new Symfony\Component\HttpFoundation\RedirectResponse('www.myurl.com');
    $response->send();
    return;
  }

}

Answer 1

Concerning CDNs or Varnish you need to ask in a different place. In Drupal you find two caches for requests/responses:

1. The Internal Page Cache, which is implemented in a kernel middleware, and you need to define a middleware with a higher priority to handle the uncached request.
2. The Dynamic Page Cache, which is implemented in a kernel request subscriber, and you need an event subscriber with a higher priority to handle the uncached request.

By the way, when you implement one of these methods, don't send the response, return it or set it in the event.

Example for a redirect middleware: https://drupal.stackexchange.com/a/247478/47547