Two suggestions to improve caching:

 1. In case an entity exists you have to return a neutral access result with the correct cache data.

2. The custom cache tag can be replaced by the list tag ENTITY_TYPE_list, which is invalidated automatically when entities are added or deleted.

The resulting code:

      public function access(AccountInterface $account) {
        if ($entity_does_not_exist) {
          return AccessResult::forbidden()->addCacheTags(['my_entity_list']);
        }
        return AccessResult::neutral()->addcachetags(['my_entity_list');
      }

postDelete() and postSave() is no longer necessary.