Once i tried to find some function that could do it. But i found just
fivestar_get_votes($entity_type, $id, $tag = 'vote', $uid = NULL). It returns full voting score.
You can get some info here or on other sites.
I think it is possible with some sql queries EG:
$query = db_select('votingapi_vote', 'vote');
$query->condition('vote.entity_type', 'node');
$query->condition('vote.entity_id', $nid);
$query->condition('vote.tag', 'vote');
$query->condition('vote.value', '60');
$query->addField('vote', 'uid');
$three_stars = $query->countQuery()->execute()->fetchField();
Here $nid is your your node's nid and 60 means three stars (20 - 1 star, 40 - 2 stars etc), vote.tag is tag that you assigned for your votes.