6

Drupal is moving to support utf8mb4, however, it is using utf8nb4_general_ci. Why take the time to move over to support it, and then not fully support it? Is there a specific reason, or just continuing utf8_general_ci from previous?

Would there be any problems with ignoring this and using unicode anyway?

Drupal Ticket: MySQL driver does not support full UTF-8 (emojis, asian symbols, mathematical symbols)

Here is a question on stack exchange which says really there is no reason not to use unicode now days and finally, on this question, it says "utf8_general_ci is a legacy collation that does not support expansions, contractions, or ignorable characters."

I note that WordPress uses utf8mb4_unicode_ci.

8

It seems to me that the recommendation is outdated and that utf8mb4_unicode_ci will work without problems. It has been used by a lot of people for a long time.


There is a difference between changing the character set from utf8 to utf8mb4 (to support more codepoints) and changing the collation from general_ci to unicode_ci (to get more accurate sorting). Both changes can cause their own problems, so doing both independently makes sense.

utf8mb4 is used by default since 8.0.0-beta12. The main issue seemed to be a change of key lengths limitations for InnoDB but as I understand it, utf8mb4 should have worked with the default MyISAM engine even before that change.

Switching to unicode_ci shouldn't cause problems, but may unexpectedly changes the order of sorting for some sites.

The default collation setting is just a default and modules can choose their own collations anyway if they need to. I also haven't found any documentation that says modules should expect a certain collation. The database install guide just lacks a clear statement about which collations are supported and is inconsistent:

  • In the section about phpMyAdmin it says that you have to

    Make sure you select COLLATION utf8_general_ci

  • Later in the section about installation from command line, general_ci doesn't seem to be required and any UTF-8 collation will do:

    Note: The database should be created with UTF-8 (Unicode) encoding, for example utf8_general_ci.

Furthermore, PostgreSQL is supported and it seems its default UTF-8 collation is equivalent to utf8mb4_unicode_ci, so using that with MySQL should be fine too.

  • Awesome, thank you for helping me understand this! – Paul Dec 21 '15 at 15:56
1

Below link to post explains than utf8mb4_unicode_ci is better than utf8mb4general_ci (which is a little bit faster) because the second one have problems in sorting order in some languages: https://stackoverflow.com/a/766996/860099

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.