如何Handle Byte Order Mark (BOM)

问题

你的文本 file 开始具有不可见 BOM bytes 该原因 parsing 错误或 出现 as strange characters 在输出.

前置条件

• Understanding 的字符 encodings
• 熟悉hex editors 或字节 inspection

步骤

1. 1. Identify the BOM bytes

   Inspect the first 2-3 bytes of the file. EF BB BF indicates a UTF-8 BOM, FF FE indicates UTF-16 LE, and FE FF indicates UTF-16 BE. The BOM tells the reader the encoding and byte order.

   输入

   EF BB BF 3C 3F 78 6D 6C

   输出

   UTF-8 BOM + '<?xml'

2. 2. Decide whether to keep or strip it

   Keep the BOM when writing UTF-8 files for Windows applications that rely on it for encoding detection. Strip it for web files (HTML, CSS, JS, JSON) where the BOM can cause parsing errors or appear as a visible character.

   输入

   JSON file with BOM

   输出

   strip BOM before parsing

3. 3. Strip the BOM programmatically

   When reading a file, check for and remove the BOM before processing. Most modern parsers handle this automatically, but some (like JSON.parse in older environments) do not and will throw an error on a BOM.

   输入

   \uFEFF{}

   输出

   strip → {}

预期结果

Text files 具有 BOMs handled 正确为 其目标上下文: kept 为 Windows compatibility, stripped 为 web 和 API use.

常见错误

JSON.parse fails with 'Unexpected token'

The file has a UTF-8 BOM. Strip the first 3 bytes (EF BB BF) or the U+FEFF character before parsing.

BOM appears as  in HTML output

The BOM was not stripped and is rendered as a zero-width no-break space. Remove it from the source file.

排障指引

• If a file works locally but fails on a server, check for a BOM added by a Windows editor.
• Use a hex editor to confirm whether a BOM is present when debugging encoding issues.

安全限制

• A BOM is metadata, not a security feature. It can interfere with signature verification if not accounted for.