Drupal doesn't consider a folder to be a group of files, and doesn't have a method for bulk deleting managed files (like it does for unmanaged files:
file_unmanaged_delete_recursive). So you'll need to identify which files you want to delete manually. A query with
WHERE uri LIKE 'public://folder_name/%' might be sufficient in this case, for example, but you'll know you're own setup better than I can guess.
Once you have a list of file IDs, load up the objects, loop through them, and call
file_delete on each. If you have the File Entity module installed, you can shorten this by using
file_delete_multiple(), and just pass the IDs.
Once you're happy all the files have been deleted (you could stat the folder after running the loop), then remove the folder with
Bear in mind that
file_delete() is not guaranteed to remove the file; other parts of the system can declare their interest in a managed file by way of the
file_usage table. If you want to unconditionally remove files regardless of any other entity in the system that they're attached to, you can use the
file_usage_* methods in file.inc.