Unique Log Line Extractor

When you need the answer to "what distinct things are in here?", Unique Log Line Extractor delivers — a deduplicated list of every distinct line, each annotated with how many times it actually occurred, sorted by frequency so the most common entries surface first.

Whole-line dedup is the default, but two options change what "distinct" means. "Ignore leading timestamp" strips a recognized timestamp prefix (ISO-style YYYY-MM-DD HH:MM:SS, syslog-style Mon D HH:MM:SS, bracketed Apache/nginx [DD/Mon/YYYY:...], or a raw Unix epoch) before comparing lines, so a crash-loop error that repeats every second with a different timestamp collapses into one distinct entry with an honest occurrence count instead of showing up as hundreds of "different" lines. "Case-insensitive" folds casing before comparing, so Timeout and TIMEOUT are treated as the same message.

Beyond whole lines, token mode flips the question: instead of "which lines are distinct", it asks "which values matching this pattern are distinct". Supply any regular expression — an IPv4 pattern, a URL-path pattern, \d{3} for status codes — and every match across every line is extracted and counted separately. This is the fastest way to enumerate exactly which endpoints, client addresses, or status codes actually appear in a log, without writing a full parser for the log format first.

• Distinct lines with frequency counts — every unique line text, sorted by how often it occurred.
• Ignore leading timestamp — recognizes ISO, syslog, Apache/nginx bracketed, and Unix-epoch timestamp prefixes so repeating messages with different timestamps collapse correctly.
• Token mode — supply any regex (IPs, URL paths, status codes, anything) and get the distinct matched values with counts, instead of whole lines.
• Case-insensitive option — fold casing when comparing lines or tokens.
• Copyable output — value-and-count pairs as plain text, ready to paste into a report or spreadsheet.

1. Paste the log you want to deduplicate or enumerate.
2. Choose a mode — "Distinct lines" for whole-line dedup, or "Distinct tokens by pattern" to extract specific values.
3. Tune options — ignore the leading timestamp for line mode, enable case-insensitivity for either mode, and supply a regex pattern for token mode.
4. Click Extract. Every distinct value appears with its occurrence count, most frequent first.

Example 1 — Distinct endpoints. Token mode with a URL-path pattern lists every API path your log mentions exactly once each, with a hit count — a fast way to build an endpoint inventory from traffic alone, without access to the router config.

Example 2 — Collapsing a crash loop into one message. Enable "ignore leading timestamp" on a log full of the same error repeating every few seconds with a fresh timestamp each time; instead of hundreds of "different" lines, you get one distinct message with an accurate repeat count.

Example 3 — Status codes. Token mode on \d{3} reveals the exact set of HTTP status codes present in the log — useful for confirming whether a rare code like 429 or 503 shows up at all before writing an alert rule for it.

Example 4 — Case-normalized error vocabulary. A log with inconsistent casing (Connection Timeout vs connection timeout) collapses into a single entry when case-insensitivity is enabled, giving an accurate count of how often that failure mode actually happens.

• Vocabulary view — see every distinct value in a log at a glance, not a wall of near-duplicates.
• Accurate counts — timestamp and case normalization prevent one message from being miscounted as many.
• Two extraction modes — whole lines for message-level dedup, tokens for pattern-level enumeration.
• No format-specific parser needed — any regex works for token mode, so it adapts to any log format.
• Private — extraction and counting run entirely client-side.