Why does my text show weird symbols? Why does your text show Café, ’, or �? That’s mojibake — UTF-8 bytes misread as Windows-1252 or vice versa. Your file was saved as UTF-8 (Café = bytes C3 A9), but the browser, database, or editor decoded it as Windows-1252 (C3 → Ã, A9 → ©) — so one correct byte pair becomes two wrong characters. Since 2023, 98% of the web is UTF-8, yet one missing <meta charset="utf-8">, a mis-labeled CSV, or a double-encode can still garble every accent, emoji, and quote. This guide explains what encoding actually is, the 7 patterns that reveal the exact mismatch, and how to fix mojibake in code, editors, and databases without guessing.
- One line: Text is bytes + a label (charset). If writer saves as UTF-8 but reader decodes as Windows-1252 (or vice versa), bytes are mapped to wrong characters — that’s mojibake. See W3C What Is Encoding + MDN Character Encoding.
- Spot the pattern:
é → é(UTF-8 read as 1252),“ → “(smart quotes double-mojibaked),�(replacement char FFFD for invalid bytes),????(bytes stripped). Pattern tells the exact mismatch. - Why UTF-8 wins: ASCII 0–127 = 1 byte (compatible), 128+ = 2–4 bytes (é = C3 A9, 😀 = F0 9F 98 80) per RFC 3629 UTF-8 + Unicode 15. Windows-1252 is single-byte (é = E9) — mixing them garbles.
- Fix loop (30 sec): 1) Declare UTF-8 everywhere: HTML
<meta charset="utf-8">first in<head>, HTTPContent-Type: text/html; charset=utf-8, DButf8mb4_unicode_ci, file saved as UTF-8 without BOM. 2) Convert:iconv -f windows-1252 -t utf-8 file.txtor JSnew TextDecoder('utf-8').decode(bytes)(MDN TextDecoder). 3) Re-validate: no �, no é. - Tooling: Detect via chardet / “what the heck is this” test, fix via our text tools (encoding converter) or editor (VS Code → Reopen with Encoding). See Joel Spolsky’s must-read Absolute Minimum on Unicode.
What Is Mojibake? The 5-Second Example
Mojibake is correct bytes decoded with the wrong charset — the classic Café → Café.
Try it live: save Café as UTF-8 → bytes are 43 61 66 C3 A9 (C3 A9 is é in UTF-8). Open that file as Windows-1252 → C3 maps to Ã, A9 maps to © → you see Café. Reverse: save Café as Windows-1252 → byte E9 for é → read as UTF-8 → E9 alone is invalid UTF-8 → browser shows Caf� (U+FFFD replacement). The bytes never changed — only the label did. That’s why it’s not “corruption” but mislabeling.
I see this weekly when a CSV exported from Excel (Windows-1252 by default on old Windows) is imported into a MySQL table set to utf8mb4 without conversion — every “é” becomes “é” in the web app, while emojis become “????” because Windows-1252 has no mapping for 4-byte emoji. The fix wasn’t to “clean the data” but to convert at import: iconv -f cp1252 -t utf8 data.csv.
Charset vs Encoding vs Unicode — 3 Terms in One Line
- Character set / repertoire: The list of characters (A, é, 😀) — Unicode is the universal list (149k chars in Unicode 15 per Unicode 15).
- Encoding / charset: How each character is mapped to bytes — UTF-8 (variable 1–4 bytes, ASCII compatible), Windows-1252 (single byte, 256 slots), UTF-16. See W3C Definitions and MDN.
- Mojibake: Decoding bytes with a different encoding than they were written — the mismatch, not the bytes themselves.
How Encoding Actually Works — Bytes, Not Magic
Text on disk is always bytes; the charset tells the decoder how to turn bytes back into characters. Per RFC 3629 and W3C:
| Character | UTF-8 Bytes (Hex) | Windows-1252 Bytes | What You See If Mismatched |
|---|---|---|---|
| A (U+0041) | 41 (1 byte) | 41 | Same — ASCII is identical (why English rarely mojibakes) |
| é (U+00E9) | C3 A9 (2 bytes) | E9 (1 byte) | UTF-8→1252: Café | 1252→UTF-8: Caf� |
| “ (U+201C) | E2 80 9C (3 bytes) | — (no 1-byte slot) | UTF-8→1252: “ (3 chars) |
| 😀 (U+1F600) | F0 9F 98 80 (4 bytes) | — (needs surrogate) | UTF-8→1252: 😀 or ???? |
Why UTF-8 won: it’s ASCII-compatible (English text is identical bytes in both), self-synchronizing, and covers all Unicode. W3C recommends utf-8 for all new content (W3C). Joel Spolsky’s classic “There ain’t no such thing as plain text” (Joel on Software) makes the same point: a string is bytes plus an encoding label — forget the label and you get mojibake.
Where the Label Lives — 3 Places That Must Agree
- File bytes + BOM: The file itself may start with
EF BB BF(UTF-8 BOM) — discouraged for web, but some editors add it. BOM can break JSON/CSV parsers if unexpected. - HTTP header:
Content-Type: text/html; charset=utf-8— highest priority for browsers. - Meta tag:
<meta charset="utf-8">as first child of<head>— fallback if header missing.
If these disagree, browser follows HTTP header > BOM > meta (per HTML spec). That’s why your local .html looks fine (meta says utf-8) but the same file on the server mojibakes (server sends charset=windows-1252). Check DevTools → Network → Response Headers → content-type.
Why Does My Text Show Weird Symbols? The 7 Mojibake Patterns
Each garble is a fingerprint — learn to read it and you know the exact fix without guessing.
| # | You See | Actual | Mismatch | Fix |
|---|---|---|---|---|
| 1 | Café | Café | UTF-8 bytes read as 1252 | Decode as 1252 → re-encode as UTF-8 |
| 2 | Caf� or Caf? | Café | 1252 bytes read as UTF-8 (invalid → FFFD) | Decode as UTF-8 → read as 1252 correctly |
| 3 | “Hello†| “Hello” | UTF-8 smart quotes (3 bytes) read as 1252 | Same as #1 — convert |
| 4 | 😀 or ???? | 😀 | 4-byte emoji via 1252 (no glyph) or stripped | Use UTF-8 end-to-end, utf8mb4 in DB |
| 5 | Café | Café | Double-encoded (UTF-8 → 1252 → UTF-8 again) | Encode once, decode once |
| 6 | Hello | Hello | UTF-8 BOM EF BB BF rendered as text | Save as UTF-8 without BOM |
| 7 | æ–‡å— (Chinese garbled as Latin) | 文字 | UTF-8 Chinese (3 bytes each) read as 1252 | Declare UTF-8 in HTTP + meta + DB |
Pattern 1: é — The Classic (UTF-8 → 1252)
One UTF-8 character becomes two 1252 characters. é = C3 A9 in UTF-8 → 1252 maps C3→Ã, A9→© → é. Similarly, — (E2 80 94) → —. Count: every non-ASCII becomes 2–3 garbled chars — that doubling is diagnostic. Fix in code: treat the mis-decoded string’s bytes as 1252 then decode as UTF-8. In Python: "Café".encode("cp1252").decode("utf-8") → "Café" (works because é round-trips). In JS: it’s already garbled after fetch — fix the source charset header, not the string after.
Pattern 2: � — The Replacement Character (1252 → UTF-8)
� (U+FFFD) means the decoder hit an invalid UTF-8 byte sequence and substituted. Windows-1252 é is single byte E9 — that byte alone is invalid UTF-8 (needs leading C2–F4) → decoder replaces with �. You lose information: E9 → � cannot be reversed to é — you must re-fetch with the right encoding. Check DevTools → Network → Response → preview vs raw bytes. Browser shows � when header says charset=utf-8 but body is actually Windows-1252.
Pattern 5: é — Double Mojibake (The Double Fix Trap)
Double-encode happens when you “fix” by re-encoding already-garbled text as UTF-8 again. Original UTF-8 C3 A9 → misread as 1252 → é (bytes C3 A9 interpreted) → saved again as UTF-8 → each of à (C3 83) and © (C2 A9) becomes 2 bytes → total 4 bytes C3 83 C2 A9 → read as 1252 → é. Fixing double requires two reversals: encode 1252 → decode UTF-8 twice, but better to fix source once — every extra encode worsens loss with �.
How to Diagnose in 10 Seconds — Look at the Garble
Don’t guess — the characters tell you the mismatch.
- See Ã, Â, â? → UTF-8 read as Windows-1252/Latin-1. Fix: decode bytes as 1252 then re-decode as UTF-8, or re-save source as UTF-8 and serve
charset=utf-8. - See � or ? → Invalid UTF-8 (often 1252 bytes read as UTF-8, or data stripped to
?by a 7-bit path). Check HTTP header vs meta vs DBcharset— one disagrees. - See  at start? → BOM
EF BB BF— save file as UTF-8 without BOM (VS Code → Save with Encoding). - See ???? for emoji? → DB column is
utf8(3-byte) notutf8mb4(4-byte) — MySQLutf8truncates 4-byte emoji to?. ALTER toutf8mb4. - Check headers: DevTools → Network → click document → Response Headers →
content-type. If it sayscharset=windows-1252but your<meta charset="utf-8">says utf-8, header wins — fix server config.
# Quick proof — what bytes are actually on disk?
# Linux/macOS
hexdump -C file.txt | head
# Café as UTF-8 → 43 61 66 c3 a9
# Café as Windows-1252 → 43 61 66 e9
# Python — detect and fix classic é pattern
s = "Café" # mojibaked
fixed = s.encode("cp1252").decode("utf-8")
print(fixed) # Café
# Node — fetch with wrong charset (simulate)
# Headers: content-type: text/html; charset=windows-1252 but body is utf-8 bytes
# Fix: ensure server sends charset=utf-8, or use TextDecoder
const bytes = new Uint8Array([0x43, 0x61, 0x66, 0xC3, 0xA9]);
new TextDecoder("utf-8").decode(bytes) // Café
new TextDecoder("windows-1252").decode(bytes) // Café
How to Fix It — By Where You Control It
Fix at the source (where bytes are written), not after they’re garbled — re-encoding � loses data.
Fix 1: HTML / HTTP — Declare UTF-8 Once, Everywhere
<head>
<meta charset="utf-8"> <!-- first element in head -->
</head>And HTTP header (highest priority):
# Apache .htaccess
AddDefaultCharset UTF-8
# Nginx
charset utf-8;
# Node/Express
res.setHeader("Content-Type", "text/html; charset=utf-8");Verify: curl -I https://example.com | grep -i content-type must show charset=utf-8. See W3C for why meta must be first.
Fix 2: Files — Save as UTF-8 Without BOM
VS Code: Save with Encoding → UTF-8 (not “with BOM”). Notepad++: Encoding → Convert to UTF-8 without BOM. Excel “Save as CSV UTF-8” is often actually cp1252 with BOM — verify with hexdump before importing. Our text tools converter shows byte preview so you see C3 A9 vs E9 before saving.
Fix 3: Databases — Use utf8mb4, Not utf8
MySQL’s utf8 is not UTF-8 — it’s 3-byte max, so emoji (4-byte) truncates to ? or ????. Use utf8mb4:
ALTER DATABASE mydb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE posts CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
-- For existing mojibaked rows: fix by converting via binary
UPDATE posts SET title = CONVERT(CAST(CONVERT(title USING latin1) AS BINARY) USING utf8mb4)
WHERE title LIKE '%Ã%';Postgres is UTF-8 by default — ensure client encoding matches (SET client_encoding TO 'UTF8'). See Joel on Software for why “plain text” has no meaning without encoding.
Fix 4: CSV / Excel — Convert at Import, Not After
Excel on Windows defaults to cp1252. When importing to a UTF-8 system, convert on the fly: iconv -f cp1252 -t utf-8 data.csv > data.utf8.csv or python -c "import codecs; open('out.csv','w',encoding='utf-8').write(open('data.csv',encoding='cp1252').read())". In JS, use TextDecoder with the source encoding label if you must read legacy bytes. Don’t “fix” by replacing é → é manually — you’ll miss “ and double-encode others.
Fix 5: Double Mojibake — Stop Re-Encoding Garbled Text
If you already saved mojibaked text to DB, don’t run a blanket replace — it double-encodes the é that was once é. Instead, restore from pre-mojibake backup or reverse once: SELECT CONVERT(CAST(CONVERT(col USING latin1) AS BINARY) USING utf8mb4) for the single-mismatch case. If é (double), apply twice but test on a copy first — �-replaced rows are irreversible.
Why Copy-Paste From Word, Excel, and PDFs Breaks Text
Word, Excel, and PDFs don’t use plain ASCII quotes — they use smart quotes, em dashes, and non-breaking spaces that are 2–3 byte UTF-8 sequences invisible until they mojibake.
Word’s “Hello” is not "Hello" — it’s E2 80 9C 48 65 6C 6C 6F E2 80 9D (left/right smart quotes are 3 bytes each). So is Excel’s non-breaking space C2 A0 (looks like a space, but it’s not 20) and PDF’s ligature �. When you copy that into a system that expects Windows-1252 or strips to ASCII, each smart quote becomes “ → 3 garbled chars per quote. I see this after every “Paste from Word” into a CMS that saves as Latin-1 — the article looks fine in Word, then shows — for every em dash on the live site.
Fix: paste via “Paste as plain text” (Ctrl+Shift+V) or run through a plain-text step: Word → Notepad (which forces UTF-8) → CMS, or use our text tools “Smart quotes → straight quotes” converter that maps E2 80 9C → 22 and C2 A0 → 20 before saving. In code, normalize on input: Python text = unicodedata.normalize('NFKC', text).replace('\xa0',' ') — that NFKC folds ligatures and compatibility chars back to ASCII where appropriate.
PDF copy is worse: PDF text extraction often inserts � for unmapped glyphs and line-break hyphens. If you extract with pdftotext -enc UTF-8 without that flag, you get Windows-1252 bytes labeled as UTF-8 → mojibake on every accented word. Always specify encoding at extract time and verify with hexdump, not by eye.
Best Practices Checklist — Keep Text UTF-8 End-to-End
| Layer | Set To | Don’t |
|---|---|---|
| File | Save as UTF-8 without BOM (VS Code) | UTF-8 with BOM for JSON/CSV (breaks parsers) |
| HTML | <meta charset="utf-8"> first in head | No meta or meta after 1024 bytes |
| HTTP | Content-Type: ...; charset=utf-8 | Server default charset=iso-8859-1 |
| DB | MySQL utf8mb4_unicode_ci / Postgres UTF8 | MySQL utf8 (3-byte) for emoji |
| Code | Read with explicit encoding: decode('utf-8') | Rely on locale default (open().read() without encoding) |
| CSV | Export/Import as UTF-8, verify with hex | Excel cp1252 → utf8mb4 without convert |
Automate: add file --mime-encoding *.csv to CI, and in JS always create text via TextEncoder('utf-8') rather than concatenating bytes manually. For quick fixes, paste garbled text into our text tools encoding converter — it shows byte preview (C3 A9 vs E9) so you see the mismatch before saving.
Practice Lab — Fix Mojibake in 2 Minutes
# Lab — no install
1) Create a file with Café as UTF-8: echo -n "Café" | hexdump -C # C3 A9
2) Misread as 1252: python -c "print(open('file.txt',encoding='utf-8').read().encode('utf-8').decode('cp1252'))"
# → Café (classic pattern)
3) Fix in Python: python -c "print('Café'.encode('cp1252').decode('utf-8'))" # → Café
4) Check your page: DevTools → Network → Headers → content-type charset → must be utf-8
5) Fix at source: HTML <meta charset="utf-8"> first in head + HTTP header + DB utf8mb4
# You just proved: same bytes (C3 A9) → decoded two ways → mojibake vs correctYou just reproduced the exact byte-level cause that makes every accent double — and the one-line fix that reverses it for the classic pattern.
Frequently Asked Questions
Why does my text show é instead of é?
That’s UTF-8 bytes C3 A9 (é) read as Windows-1252, where C3→à and A9→©. Your file was saved as UTF-8 but the browser/server decoded it as 1252. Fix: ensure the source sends charset=utf-8 (meta + HTTP header) and re-save the file as UTF-8. In Python, "Café".encode("cp1252").decode("utf-8") reverses the classic case. See W3C.
Why do I see � (black diamond question mark)?
� is U+FFFD, the Unicode replacement character — the decoder hit an invalid byte sequence for the declared charset (often Windows-1252 bytes read as UTF-8, where single byte E9 is invalid). Unlike é, this one loses data: E9 → � can’t be reversed. You must re-fetch with the correct encoding label instead of re-encoding the � string. See MDN TextDecoder.
What’s the difference between UTF-8 and Windows-1252?
UTF-8 is variable-length (1–4 bytes, ASCII compatible) and covers all Unicode via RFC 3629; Windows-1252 is single-byte (256 characters, Western Europe) where é is one byte E9. UTF-8 é is two bytes C3 A9 — mixing them causes mojibake. Use UTF-8 everywhere new; reserve 1252 only for legacy files you convert. See MDN.
How do I fix mojibake in MySQL where emoji shows as ????
MySQL’s utf8 is 3-byte and truncates 4-byte emoji (F0 9F 98 80) to ?. Convert to utf8mb4: ALTER DATABASE db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; ALTER TABLE t CONVERT TO CHARACTER SET utf8mb4;. For existing mojibaked rows with é, run CONVERT(CAST(CONVERT(col USING latin1) AS BINARY) USING utf8mb4) once — test on a copy because � rows are irreversible.
Can I just replace é with é to fix it?
Don’t — that fixes one character but misses “, —, emoji, and double-encoded é. Blind replace also double-encodes already-fixed text. Use a proper converter: bytes C3 A9 (UTF-8) vs E9 (1252) — tools that show hex preview let you verify. Our text tools converter previews bytes so you confirm before saving.
Why does clearing cache not fix mojibake?
Because mojibake is an encoding label mismatch, not a cached file — the bytes are correct but decoded wrong. Clearing cache re-downloads the same bytes with the same wrong label, so garble returns. Fix the label: HTTP charset=utf-8 or <meta charset="utf-8">, not the cache.
How do I know what encoding my file actually is?
Use file --mime-encoding file.txt (Linux/macOS) or hexdump: Café as UTF-8 is 43 61 66 C3 A9, as Windows-1252 is 43 61 66 E9. In VS Code, status bar shows encoding — click to “Reopen with Encoding” to test. Browser DevTools → Network → Headers shows what charset the server claimed.
Should I save files as UTF-8 with BOM?
No for web/JSON/CSV — BOM EF BB BF renders as  at the start (pattern #6) and breaks JSON parsers. Save as UTF-8 without BOM (VS Code: Save with Encoding → UTF-8). Use BOM only if a legacy Windows tool requires it.
Last updated: September 2, 2026 • Author: Toolwasp Team • Sources verified Sep 2, 2026: W3C What Is Encoding, MDN Character Encoding, MDN TextDecoder, RFC 3629 UTF-8, Unicode 15, Joel on Software Unicode, W3C Definitions, Toolwasp Text Tools.
