The utf16
character set is the
ucs2
character set with an extension that
enables encoding of supplementary characters:
For a BMP character, utf16
and
ucs2
have identical storage
characteristics: same code values, same encoding, same
length.
For a supplementary character, utf16
has a special sequence for representing the character
using 32 bits. This is called the “surrogate”
mechanism: For a number greater than
0xffff
, take 10 bits and add them to
0xd800
and put them in the first 16-bit
word, take 10 more bits and add them to
0xdc00
and put them in the next 16-bit
word. Consequently, all supplementary characters require
32 bits, where the first 16 bits are a number between
0xd800
and 0xdbff
,
and the last 16 bits are a number between
0xdc00
and 0xdfff
.
Examples are in Section
15.5
Surrogates Area of the Unicode 4.0 document.
Because utf16
supports surrogates and
ucs2
does not, there is a validity check
that applies only in utf16
: You cannot
insert a top surrogate without a bottom surrogate, or vice
versa. For example:
INSERT INTO t (ucs2_column) VALUES (0xd800); /* legal */ INSERT INTO t (utf16_column)VALUES (0xd800); /* illegal */
There is no validity check for characters that are technically
valid but are not true Unicode (that is, characters that
Unicode considers to be “unassigned code points”
or “private use” characters or even
“illegals” like 0xffff
). For
example, since U+F8FF
is the Apple Logo,
this is legal:
INSERT INTO t (utf16_column)VALUES (0xf8ff); /* legal */
Such characters cannot be expected to mean the same thing to everyone.
Because MySQL must allow for the worst case (that one
character requires four bytes) the maximum length of a
utf16
column or index is only half of the
maximum length for a ucs2
column or index.
For example, in MySQL 5.5, the maximum length of
a MEMORY
table index key is 3072 bytes, so
these statements create tables with the longest allowed
indexes for ucs2
and
utf16
columns:
CREATE TABLE tf (s1 VARCHAR(1536) CHARACTER SET ucs2) ENGINE=MEMORY; CREATE INDEX i ON tf (s1); CREATE TABLE tg (s1 VARCHAR(768) CHARACTER SET utf16) ENGINE=MEMORY; CREATE INDEX i ON tg (s1);
User Comments
Add your own comment.