I have several OG sites in which the groups structure group content using either taxonomy terms or nodes as categories. In my experience, working with nodes is easier than taxonomy in OG sites, but it depends on your specific purpose. Out of the box non-admin user can't create terms via admin-interface, while they can create nodes, thus also category-like nodes.
In short, using nodes: create a content type 'category' and set it as being group content.
Use a entity reference field in all content types, referencing that content type.
Create a view that lists only category nodes part of currently viewed group.
In the category content type ER field settings, entity selection, set your view you just created.
The same applies to terms:
I quote https://www.drupal.org/node/1507608#comment-8178015
Create a group audience for your vocabulary
Edit the created field and within "GROUPS AUDIENCE FIELD OPTIONS", set "Node", "Organic groups"
Go to Views and create a "Term View"
Set "Format" to "Entity Reference list"
Set "Format options" to the field you need (Taxonomy name for example)
Add a "Filter Criteria" "Vocabulary name" to filter terms
Add a "Contextual Filter" "OG membership: Group ID". There is no context since we are creating a node, so tick "Provide default value" > "The OG groups of the logged in user". Leave the other two settings "Group type" and "Concatenator".
Go to your content type and create an "Entity Reference" field
In "Field options" set "Target type" to "Taxonomy Term"
In "Entity selection" set "mode" and select your View. I leave arguments empty
Create your node and check how taxonomies are filtered
Hope it helps someone :)
If your user can't create the term via admin-interface, perhaps rules can help to set the group audience. It's only a thought...