The configuration entity is used to create configuration items. Entities allow you to store more complex configuration structures, which can be listed, edited, deleted, etc., but, above all, can be instantiated multiple times.
Examples of configuration entities include date formats, vocabularies, image styles, roles, menus, views, Pathauto URL patterns, and text formats. All have in common that you can create multiple such elements, and perform other common operations like edit and delete. They also have in common that they are entities that are only managed from the configuration or administration of the site, without exposing them to the end user.
Configuration entities are translatable and can provide default values that will be taken into account during installation. Configuration entities can not have fields.
Content Entity. Content entities are used to create content items. They differ mainly because they have fields. The Manage fields, Manage presentation, etc. options are common in content entities. They are also translatable and, optionally, may have revisions.
Examples of content entities are nodes, comments, taxonomy terms, users, files, custom block types, etc.
A fundamental difference between the two variants is that the configuration entities use the configuration system to store the information, instead of the database, which is the storage used by the content entities.