Character Frequency Analyzer

Character frequency analysis counts how often each character appears in a piece of text and what share it represents. It can cover letters, digits, punctuation, whitespace, CJK characters, symbols, and unusual control characters depending on the input. Typical work includes cryptography exercises, simple substitution analysis, text feature extraction, data cleaning, encoding checks, compression research, and quality control on imported content. Unusual frequencies can reveal hidden formatting characters, mojibake, repeated separators, unexpected languages, or copy-paste artifacts. The tool does not explain every cause by itself, but it gives a measurable view of the actual character distribution so the next inspection step is clearer.

Character frequency analysis counts the occurrences of each character in a text. The implementation uses a hash table (typically a Map in JavaScript): 1) traverse the text character by character, 2) for each character, increment the count in the map, 3) sort the entries by count descending, 4) render the histogram (top 10-20 characters, with the rest in a tail bucket). The algorithm is O(N) in the length of the text, where N is the number of characters, and the dominant cost is the map lookup and increment (essentially free in V8 and SpiderMonkey for the common case). A useful subtlety: what counts as a character? In JavaScript, .length returns the number of UTF-16 code units, not Unicode code points. A character outside the Basic Multilingual Plane (emoji, rare CJK ideographs) is encoded as a surrogate pair (two code units), and a naive per-code-unit count will over-count by treating each surrogate half as a separate character. The page uses Array.from(text) or Intl.Segmenter to iterate code points (or grapheme clusters), which is the right definition of a character for most use cases. A useful application: frequency analysis is the classical technique for breaking substitution ciphers (Caesar, Vigenere, simple substitution). The English letter frequency distribution is well-known (E, T, A, O, I, N, S, H, R are the most common, in that approximate order), and a substitution cipher preserves the frequency distribution, so the most common ciphertext letter likely maps to E, the second to T, and so on. The page is a teaching tool, not a cryptanalysis tool, but the technique is the same. A caveat on Unicode: for CJK languages, the character frequency distribution depends on the corpus (modern novel, classical poetry, technical text) and the level of analysis (character, word, bigram, trigram). A frequency analysis of a modern Chinese novel gives one distribution; a frequency analysis of the Analects gives a different one. The page is corpus-agnostic, so the user can run it on any text.

• Hash table (Map in JS): traverse text, increment count, sort by count desc, render histogram; O(N) in text length with very fast per-character cost.
• Surrogate pairs: a character outside the BMP (emoji, rare CJK) is two UTF-16 code units; naive per-code-unit count over-counts; use Array.from or Intl.Segmenter.
• Substitution cipher cryptanalysis: English letter frequency E, T, A, O, I, N, S, H, R (most common in that order); a substitution cipher preserves the distribution.
• CJK frequency: depends on the corpus (modern novel, classical poetry, technical text) and the level of analysis (character, word, bigram); the page is corpus-agnostic.
• Top-N and tail: the page renders top 10-20 characters by default, with a tail bucket for the rest; useful for spotting patterns in dense text.
• Case sensitivity: the page exposes a case-sensitive/insensitive toggle; English is typically case-insensitive (E and e are the same letter, just different shapes), CJK is always case-insensitive (no concept of case).
• Performance: O(N) for counting, O(K log K) for sorting K distinct characters; the page handles 1M-character texts in well under a second, which is the production pattern.