WL#2637: Byte Order Mark for LOAD DATA INFILE and mysqlimport

Sometimes Unicode files begin with a Byte Order Mark (BOM). 
We want to ignore the BOM when reading files using LOAD 
DATA INFILE or mysqlimport. 
 
We could support a new clause "IGNORE n BYTES" 
(analogous to "IGNORE n LINES") but that would 
not work well: sometimes there might be a BOM, 
sometimes not, and we only want to ignore the 
bytes if they are there. 
 
For a new LOAD DATA INFILE clause, we'll use the 
Oracle10g SQL Loader syntax "BYTEORDERMARK CHECK | NOCHECK", 
It's described in the Oracle10g Utilities Manual: 
http://dbis.informatik.uni-freiburg.de/oracle-docs/doc1001/server.101/b10825/ldr_field_list.htm#i1011032

(Search for the word "BYTEORDERMARK".) 
We won't worry about Oracle's clause order, though, 
so put "BYTEORDER" clause after "INTO TABLE tbl_name". 
 
The default is "BYTEORDERMARK NOCHECK". 
Thus, if there is some MySQL-supported character set 
where the BOM characters are meaningful, no problem. 
(This is not how Oracle handles the default.) 
 
For mysqlimport, the flag is --byteordermark=check 
or --byteordermark=nocheck (the default) 
 
SELECT ... INTO OUTFILE does not generate BOMs. 
 
"BYTEORDERMARK CHECK" doesn't really mean that we 
check. All it means is: we skip the first 2 or 3 bytes of 
the file if they are equal to any of the following: 
0xEFBBBF                              UTF8 
0xFEFF                                UCS2 bigendian 
0xFFFE                                UCS2 littleendian 
Even if BYTEORDERMARK CHECK is on, the absence of a BOM 
will not be an error. Just treat the first bytes as data. 
 
See also: WL#993 "LOAD DATA INFILE and character sets". 

Feature requests:
BUG#4960 Mysql cmdline client fails on byte order mark

See also:
http://bugs.mysql.com/bug.php?id=10573
http://bugs.mysql.com/bug.php?id=29323