There are different parts to this answer.
since it doesn't use the core theme_breadcrumb() function.
By default, it does use theme('breadcrumb').
But you should have a look at the settings in admin/structure/crumbs/display.
But the check_plain() that causes special characters to be double-escaped happens in theme_crumbs_breadcrumb_link() and theme_crumbs_breadcrumb_current_page(), which you could also override.
You could also look at the $double_encode option in htmlspecialchars(), so you could use your own version of check_plain().
This being said:
Simply omitting the sanitization for all breadcrumb links is a bad idea, and potentially exposes your site to script injection. If anything coming from a non-trusted source ends up in the breadcrumb, or if people add broken html into a node title or menu link title or elsewhere, this could cause broken or malicious html on your site.
I think the better option is to just use the literal character in the node title, instead of the html entity.
Some general info:
With Crumbs, every breadcrumb item title is provided by a plugin. The plugins take these from menu links, from the title / title callback of the router item, or elsewhere. In most cases there is already an existing plugin which provides the breadcrumb item title you need.
Otherwise you can write your own plugins. I can give you a hint about that, but I need more information where you want the special char to show up. (I will edit the answer then)