Encoding Issue Detector

When a log passes through a text pipeline, encoding mistakes leave readable fingerprints: the U+FFFD replacement character where bytes could not be decoded, mojibake like é where a UTF-8 byte sequence was decoded twice, stray NULL or control bytes, or a leading UTF-8 BOM that quietly breaks the first field of every record. Encoding Issue Detector scans a log line by line for all four problems and reports every offending line with exactly what it found, entirely inside your browser.

Built-in mojibake recognition covers the common double-decoded byte patterns you actually run into: accented Latin letters like é and ñ that were UTF-8 bytes decoded a second time as Latin-1 or Windows-1252, and the three-byte sequences UTF-8 uses for curly quotes, apostrophes, and em dashes when those get mangled the same way. Alongside mojibake, the scanner checks for the raw U+FFFD replacement character decoders insert whenever they hit bytes they cannot map at all, for NULL and other non-whitespace control bytes that indicate binary data leaking into a text stream, and for a UTF-8 BOM (U+FEFF) sitting at the very start of the pasted text. A separate suspect-pattern check looks for runs of the same mojibake-style byte sequence repeated back to back, which tends to show up in more severely corrupted text than an isolated mojibake hit.

Every finding is listed with its line number, the specific marker or markers found on that line, and a preview of the line itself, so invisible corruption becomes an actionable, line-by-line list instead of a mystery. The summary counts — lines scanned, total issues, and affected lines — are kept in sync with the report: a line that trips more than one detector (say, both a replacement character and a stray control byte) counts once toward affected lines but each individual marker still counts toward the issue total, and the leading-BOM finding is reconciled with any other finding on line 1 so that one physical line is never double-counted as two affected lines.

• Replacement-character detection — the U+FFFD marker left by a failed byte-to-character decode.
• Mojibake recognition — common double-decoded UTF-8 patterns for accented letters, quotes, and dashes.
• Suspect byte-pattern check — flags repeated mojibake-style sequences and misdecoded BOM byte triples.
• NULL and control byte flags — binary pollution hiding inside text.
• UTF-8 BOM detection — a leading invisible byte sequence that breaks the first field.
• Categorized, line-numbered report — every finding shows its line and its marker type.
• Consistent stats — the affected-lines count matches what the report actually lists, with no double-counting.
• 100% local — nothing leaves the browser.

1. Paste your log into the text box, exactly as you received it.
2. Pick which detectors run — BOM, replacement characters, control bytes, mojibake, and suspect patterns are all on by default and can be toggled off individually.
3. Scan. Every line is checked against every enabled detector in one pass.
4. Review findings — each issue shows its line number and the specific marker or markers found there.
5. Copy the findings report for the pipeline owner or the ticket documenting the corruption.

Example 1 — Double-decoded ingest. A file shows é everywhere an “e” with an accent should be. The detector lists every affected line, proving the exporter decoded UTF-8 bytes as Latin-1 once too many times before saving the file.

Example 2 — Replacement-char corruption. A parser that silently dropped UTF-8 validation leaves the U+FFFD replacement character scattered across a range of records; every one of those lines is flagged individually so you can see how widespread the damage is.

Example 3 — BOM breakage. A downloaded log starts with a BOM and the first field of the header row keeps failing to match in a downstream parser. The detector calls out the BOM on line 1 immediately, and if that same first line also happens to contain a control byte, both findings are reported together against the one line rather than as two separate “affected lines”.

Example 4 — NULL byte pollution. A binary payload leaked into a text log, embedding a NULL byte mid-line. The control-byte detector flags the exact line and reports its hex code point so you know it is a NUL rather than some other stray control character.

• Visible encoding damage — turned into a concrete, line-by-line list instead of a vague sense that something looks wrong.
• Mojibake-aware — recognizes real double-decoded UTF-8 patterns, not just generic non-ASCII bytes.
• Category per finding — know exactly what you are fixing on each flagged line.
• Trustworthy stats — the affected-line count always matches what the report actually shows.
• Local and fast — no upload, instant scan, works on logs you cannot paste into a third-party site.
• Private — all processing stays local to your browser.