If I am logged in as an administrator, I get the contextual edit buttons an all blocks and nodes and can edit. If I am logged in as a user with less permission, but still enough to edit that content, those buttons aren't there. There are also no edit buttons on the user profile pages for any types of user, so if you are not an admin, you cannot easily edit this. I was hoping for a way to either turn on the contextual admin style pencil buttons for all users with permission to edit a specific bit of content, or at least be able to add my own buttons in the node or user template if allowed.
Contextual Links are a Permission that can be enabled for different roles via admin/people/permissions. If enabled, it will only show the options that user has access to.
If I wanted to add my own custom buttons in different situations, I'd start with Display Suite which allows me to create custom fields (which might be my buttons) and show/hide those buttons based on View Mode, content type, etc.