It is a bit complex but I'll try to help get things started.
Since the view is just a dynamic collection of content it makes sense to monitor the content for the types of events/changes that affect the view result.
The module you'll need for this is Rules. I see from your profile that you have some experience here.
The Fivestar module doesn't appear to have any Rules integration so you may need to create a custom Rules condition to distinguish comments with votes. (Or maybe not - I'm just not familiar with the module.)
At its simplest, it boils down to triggering an action based on the criteria you have described above. The tricky part is checking the new vote count against the view result so that you can update the field accordingly.
One way you may be able to do this is to do a before-and-after check when a comment is saved or updated. Rules Bonus Pack (in particular the submodule "rules_misc") has some Views-related additions that come in handy here.
A rough outline of a rule that alters (unsets?) the value of the field on "top 3" nodes before the comment is saved.
EVENT
Before saving a comment (of type X)
ACTION
Load a node list with Views (offered by Rules Bonus Pack)
LOOP through list
Fetch entity by id
Set a data value (entity-fetched:fieldname)
END LOOP
After the comment is saved, reset the field value on all "top 3" nodes which presumably have been recalculated based on this new data (the saved/updated comment).
EVENT
After saving a comment (of type X)
After updating an existing comment (of type X)
ACTION
Load a node list with Views
LOOP through list
Fetch entity by id
Set a data value (entity-fetched:fieldname)
END LOOP
Note: You could do all of this by storing the result somewhere and checking on cron run or when a view is being built or a node is being viewed or what have you, but by attaching the action to the events that affect the result (comment being saved/updated) you ensure the change is implemented without delay.
For the third criteria ("a node is older than 5 days") you can set up another rule that checks a condition during a cron run and alters the field on all nodes that are older than 5 days. This will obviously have some delay (depending on how often your cron runs); maybe there's a better way but I can't think of one at the moment. You would probably use Views Bulk Operations for this. Also look at Rules Scheduler (a submodule of Rules). This isn't an uncommon scenario so a quick internet search should yield several methods/tutorials.
Edit: Added more specifics on how to implement with Rules (and Rules Bonus Pack) when a comment is created/updated.