I want to load term fields in my application. Can I use entity metadata wrapper with reference to terms?
Yes. Like so:
$term_w = entity_metadata_wrapper('taxonomy_term', taxonomy_term_load(TERM_ID));
The first parameter is the entity type ID. If you visit MODULE_NAME_entity_info(), you can see the key of the definition will match the ID you need to load it.
Sam152 is right, but you don't actually need to convert the term_id to an object, it will work with just the term_id being passed as is:
$term_w = entity_metadata_wrapper('taxonomy_term', TERM_ID);