text-henkan

Remove Duplicate Lines / Sort

Clean up email lists, keyword lists and log excerpts: drop duplicates and sort the remaining lines. The tool reports how many lines were removed so you can sanity-check the result.

or choose a file

Sort modes

Ascending and descending by code point, natural order (item2 before item10, comparing digits numerically), by line length, or simply reversed. Natural order is what you want for filenames and numbered keywords.

Pre-processing options

Trimming whitespace makes lines that differ only by a trailing space compare equal. Dropping blank lines also removes whitespace-only lines. Case-insensitive comparison treats Example@x.com and example@x.com as one.

Verifying the result

After conversion you get the number of removed lines plus the most frequent duplicates with their counts, so you can confirm nothing was over-collapsed.

Frequently asked questions

Which duplicate is kept?

The first occurrence; later ones are removed. Sorting is applied after deduplication.

With case-insensitive matching, which spelling survives?

Comparison is lower-cased but the output keeps the original spelling of the first occurrence.

What is natural order?

Digits inside strings are compared as numbers, so file2 comes before file10 instead of after it.

Is there a line limit?

No. There is no server involved, so hundreds of thousands of lines work as long as browser memory allows.

Related tools

Line ending converterFull-width ⇄ half-widthCharacter counterUnicode normalizerShift_JIS → UTF-8UTF-8 → Shift_JISEUC-JP → UTF-8UTF-8 → EUC-JP