I have uploaded a large size folder via winscp to a external hosted server to the files-private folder in drupal. when i try to access the index file inside the folder in browser i get access denied. Is this because there is no entry in file_managed table ? however i could access the URL by doing same setup on a dev server without any issues.
With the private file system you cannot link direct to the file path because navigation of the private files directory is prohibited.
You have to link to drupal's private file handling url, which will then check that the current user has the correct permissions to view the private file and if so grants them access and gives them the file.
To utilise this you need to use drupal's system/files path.
So if your private files directory is files-private and you have a file files-private/filename.txt you would link to /system/files/filename.txt and drupal will handle the rest.
[Edit] The above is still valid information, however to answer our question, yes the files need to be in the file_managed table otherwise file_file_download(), which handles private file downloads, will skip over your files because it uses file_load_multiple() to try to load your file from the database.
file_load_multiple() uses entity load and system_entity_info() defines the database table file file entities as file_managed.
Thanks everyone! Finally i used https://drupal.org/project/private_files_download_permission module and sorted the problem. I could manually upload the files to files-private and this module allows access to private-files folder and we can restrict based on role.
In the question I forgot to include that we wanted to disable the file access to public and enable only for authenticated users. So this module helped to satisfy both the requirements.