Drush is a tool for performing Drupal operations in the shell. If you've already uninstalled the module then there's nothing left in Drush's purview to do, just perform a rm on each module you need removed. pm-list
has --pipe
option that will spit out the module short name. You can use that output to find the correct directory and remove it. Something like...
drush pm-list --type=Module --pipe --no-core --status="Not installed" | while read -r module; do find sites/all/modules -type d -name "$module"; done
The remaining work here is more of question about bash (or your shell of choice) than Drush so I'll leave it to the reader to find his own solution.
It should also be mentioned scripting module folder deletion could be fraught with dangers (e.g. not every module is going to be at the same folder, you shouldn't be deleting submodules, etc.); you're better off manually inspecting a list of uninstalled modules (you shouldn't have that many) and removing them judiciously.