Hopefully this is a basic question. I'm going to be setting up a drupal website there there will be multiple servers in back of a load balancer. The user will hit one web server where they will authenticate and the cookie will be added to the browser's session. On subsequent requests if the user is sent to another server, will that server recognize the user as authenticated since the authentication cookie is present, or is the cookie somehow tied to the server that authenticated the user?
Yes, if you've configured your servers correctly.
Authentication state is stored in the DB. As long as each Drupal node (app server) in your load balancer pool is configured to talk to the same database, it shouldn't matter what server the browser is talking to. Login data will persist in the database and each Drupal node should be able to find it via the user's cookie data.
The challenge is making sure all the infrastructure correctly. Those details are off topic here, but you might find Using a load balancer or reverse proxy on drupal.org helpful for other Drupal-specific stuff to know when dealing with Drupal behind a proxy (load balancer).