I would like to know what happens if a user uploads an image by
image field but doesn't save that node?
It seems that image will be in the directory but no node refers to it. Is there anyway to remove such files?
The image files will be removed during cron runs.
This happens because the
status column in
file_managed for those files will still be zero, indicating that they're temporary, and can be safely removed after a period of time.
The responsible code is in
system_cron(), in case you need to reproduce/analyse further.