You can use Select Translation.
If your view is meant to show nodes, and your nodes use the Content Translation* mechanism, you can install the Select Translation module. It provides a filter "Content translation: Select translation" by which you can define a fallback mechanism to select which language version of results should be used.
For example, you can specify that if a result is not available in the current language, its English version is used. So if no results are available for the current language, as in your case, this is equivalent to visiting the English version of the view. So effectively this "restricts the view to specific languages" resp. "hides the view in specific languages" as you asked for.
*Content Translation: That's the default translation mechanism in Drupal 7, available in Core. So if you did not install the alternative (Entity Translation), you are using this one.