You should be able to transfer the site wholesale, without too much fuss. After transferring, you may need to alter some paths (eg
admin/config/media/file-system in D7) if the
sites directory has changed paths relative to your Drupal codebase. Odds are you won't need to, though.
For a simple site, the layout will be something like this -
modules/ } don't install modules here
themes/ } nor themes here
[many other Drupal core files]
default/ } all installed code
files/ } lives in the
modules/ } site directory
themes/ } and its children
For more complex setups you may need to understand multisite layouts, but for now hopefully this answers your question.