Is there a drupal.org API for retrieving the list of available modules, a description of them and available versions?
Very recently there's an API published: https://www.drupal.org/api
There is no such API or mechanism at this time.
There is talk about extending Drush to support a Search functionality: https://drupal.org/node/1389460
But that's after a database of dependencies is created for all modules and stored in a binary-ish form fetchable by Drush: https://drupal.org/node/1299332
Turns out there is a big machine-readable list of Drupal projects available here:
Warning: Very large. :-)
Courtesy of this comment, which also tells you a URL pattern to crawl for release information: https://groups.drupal.org/node/65023#comment-204808
I wish there were good documentation about the availability of other such easter eggs.
There's an issue to Expose list of projects to external services (via JSON, XML, etc). That issue ended up providing a
tsv file with a list of all stable releases (no -dev) for all projects in drupal.org. The list is generated each 4 hours, after http://updates.drupal.org/release-history/project-list/all is regenerated.
The rationale for this file is in comment #88 if the linked issue.