Character Encoding History Reference
Reference 为遗留和 regional encodings: ASCII, ISO-8859, Windows 代码 pages, 和 EBCDIC.
Legacy Encodings
ASCII (7-bit) covers English letters, digits, and control characters (0-127). ISO-8859 family extends ASCII to 8-bit for various scripts (Latin-1, Latin-2, Cyrillic, etc.). Windows code pages (CP1252, CP936, CP932) are Microsoft extensions of ISO-8859 and regional standards. EBCDIC is IBM's mainframe encoding, incompatible with ASCII. Shift-JIS and Big5 encode Japanese and Traditional Chinese respectively. KOI8-R is a Russian encoding. These are largely superseded by UTF-8 but persist in legacy systems.
- ASCII
- 7-bit, 128 chars
- ISO-8859
- 8-bit extensions
- EBCDIC
- IBM mainframe
English onlyLatin, CyrillicincompatibleDetection and Conversion
Encoding detection is heuristic and unreliable; explicit declaration is preferred. BOM detection identifies UTF-8/16/32. Statistical methods (frequency analysis) guess legacy encodings. The WHATWG Encoding Standard defines a canonical mapping for web encodings. Convert to UTF-8 as early as possible and work in UTF-8 internally. Libraries like iconv (C), chardet (Python), and TextDecoder (JS) handle conversion. Never assume encoding; always declare or detect. Misdetected encoding causes mojibake (corrupted text).
- detection
- Heuristic, unreliable
- BOM
- Byte order mark
- mojibake
- Corrupted text
prefer explicitUTF detectionwrong encoding