Best option is to use https://drupal.org/project/imce_filefield
This is specially made to upload files to sites/default/files and add them.
Just configure imce and the field.
Uninstall file field sources, because it's not compatible to have them at the same time
You could use the FileField Sources module. To browse the server it looks like you need to use this with IMCE. From the FileField Sources project page:
The FileField Sources module expands on this ability by allowing you to
select new or existing files through additional means, including:
Re-use files by an auto-complete textfield
Attach server-side files uploaded via FTP
Transfer files from a remote server (cURL required)
Select existing files through the IMCE file browser
This question / answer was on another question:
Image Field Browser Option