I say yes since the vulnerable code is still sitting right there in the web directory which can (will) be scanned by potential attackers. Monitoring disabled modules is certainly part of our update routine.
I didn't find a whole lot of discussion on this in regards to Drupal in particular, likely because it isn't really a Drupal-related issue (see note below). Greg Knaddison (@greggles on Drupal.org) noted on a thread a few years back that:
Sometimes the vulnerabilities exist regardless of whether or not the module is enabled, so I think it's better to warn on all modules.
Source: Issue #942960: Don't warn about security updates for disabled modules
As always, best to err on the side of caution.
Note: I think this is more a server admin related question, but I hesitate to flag it as not relevant since (I dare to guess) mostIMHO people who use Drupal wouldn't see itaren't server admins shouldn't really be expected to know that way.