In Configuration > Development > Performance, in the Caching area, [Cache pages for anonymous users] and [Cache blocks] are unchecked. When I check them and save, the page reloads and both boxes are again unchecked. Is this wrong? Shouldn't they "remember" the state?
In general, yes, the form should update with the values you set. However, there are few different ways you can override form settings.
The two in question are
block_cache. You need to search the files for lines that look like:
$conf['cache'] = FALSE; $conf['block_cache'] = FALSE;
0 or the string
"0" may be used instead. Checking the settings.php for the site, and any custom modules.