Well, actually it's not too late to implement if you do the following:
1) Add a following count
(integer) field as a user field.
2) Go to /user/1/edit
and put any number other than 1, inside of this field. (this is just a temporary place holder).
3) With PHPMyAdmin or a DB viewer program, export the following count
field as a CSV file.
4) The 1 is where the USER ID goes, and the other random number is the actual following count.
So with views, create a view that shows the user ID and the following flag count number.
Then you open the exported .csv file on Excel and paste these data in the proper columns.
5) Import back into your Drupal DB and clear flush cache.
*Note: Instead of doing steps 3, 4 and 5, you could just do this with Views Bulk Operations + Rules.
6) Using the Rules module, create a rule that +1's the user following count
field, when a user flags the follow flag and -1's it when they unfollow.
7) In views, in your List of Users View, for Filter, simply add the following count
field and put less than 5.
8) Use the Field Permissions module to hide the following count
field, that way when they edit their profile, the user cannot insert/modify the number.