The answer to this question should include a "non-Drupal" part since what happens when you click the "Back" button is browser-related.
What happens on the browser
I'm not an expert on the subject, but basically all browsers add the last request that you did in a "history cache" and pressing the "Back" and "Forward" buttons moves you backward or forward in this "cache".
In general, AJAX requests are not included in this cache but this is browser specific (I think that I have read somewhere that Safari in some cases does store AJAX requests). You can find various resources on how this works for different browsers if you look it up on google (keywords could be: "back button ajax calls", along with the name of the browser that you are looking for).
Views' "Remember last selection" setting
For your specific question there are 2 things that you should notice:
- Views do allow you to store the last selection by the user in the Exposed Filters' settings. The setting's name is: "Remember the last selection". This setting also allows you to limit the roles for which the last selection will be remembered.
- The browser does not really care what Views do and, at least in the case of Chrome (probably other browsers do the same too), returns its cached version of the page (aka minus ajax calls).
You can test the above-mentioned assertion by setting the last selection option, hitting the back button (you will see the browser-cached version of the page) and then hitting the refresh button (browser will request the page again and views will kick-in and enforce the application of the last used filters).
Proof of concept
I have quickly created a dev site that I can use for my answers here and added 50 nodes with devel generate. You can see a sample view here that has AJAX enabled. After you have filtered the view using the "Body contains" exposed filter with a sample string like "oppeto" you will instantly see what Views AJAX History does: the call is being made through AJAX but the URL on your browser changes to include the exposed filters' values and is stored in your browser's history cache.
Now, when you click on a node link and then hit the "Back" browser button, you get returned to the correct, filtered version of the view (tested it on Chrome but you can test it on any browser that you want to make sure that it works as expected).
I have used this module on a couple of sites that I have built and the thing that bothered me the most is the fact that it has a known, unsolved conflict with the core Overlay module. It was not a very big deal for me since I could afford to disable Overlay at these specific cases but you should definitely consider this if you rely heavily on Overlay. For example, if you rely on Overlay Paths to show colorbox-like popups, you will find yourself in a world of hurt.
I am considering using Views AJAX History on one of my core projects but in order to do this, I will have to take a stab at the issue that I mentioned before. In any case, you can monitor the progress of the specific issue or even try to create a patch yourself.