ZapKit

Remove Duplicate Lines

Paste a list of names, emails, keywords or IDs and get it back with every repeated line removed, in its original order.

What does a duplicate line remover do?

A duplicate line remover takes a list with one entry per line and returns it with every repeat removed. It's the quickest way to clean up an email list before a mail merge, merge keyword lists from several tools, de-duplicate product SKUs or invoice numbers, or tidy a list of names collected from a form.

You decide what counts as a duplicate. Lines can be compared exactly, without regard to capital letters, or after trimming spaces from both ends, and blank lines can be dropped at the same time. The tool reports how many lines went in, how many are unique and how many were removed.

How to remove duplicate lines from a list

  1. 1.Paste the list

    One item per line, straight from a spreadsheet column, a text file or an email.

  2. 2.Choose how strict to be

    Ignore capitals so Apple and apple count as the same, and ignore stray spaces at either end.

  3. 3.Copy the clean list

    Copy or download it. The counts show how many lines were dropped.

Example: a fruit list with repeats

Nine lines: banana, Apple, cherry, banana, a blank line, apple, Date, cherry, banana. With the default settings:

Lines in
9
Unique lines
5: banana, Apple, cherry, apple, Date
Removed
4 (two bananas, one cherry, the blank line)
With case ignored too
4 lines; apple is dropped as a copy of Apple

Apple and apple are kept apart until you tell the tool to ignore case.

The first copy wins

The list keeps its order. When a line appears again further down, the later copy is dropped and the first one stays where it was, so a ranked or dated list stays in sequence.

With case ignored, the version you see is the first one written, so if Apple comes before apple you keep Apple. Trimming spaces matters more than people expect: data copied from web pages and PDFs often has invisible trailing spaces that make two identical-looking lines different.

Frequently asked questions

Does it change the order of my list?

No. Lines stay in the order you pasted them; only repeats are taken out. Use the sort lines tool if you also want it alphabetical.

Can it ignore upper and lower case?

Yes. Tick Ignore upper and lower case and lines that differ only in capitals are treated as duplicates.

How many lines can I paste?

Hundreds of thousands. Everything runs in your browser, so the limit is your device's memory, not a server.

Can I remove duplicate rows from Excel this way?

For a single column, yes: copy the column, paste it here, then paste the result back. For whole rows, copy the rows; each row becomes one tab-separated line.

More Utilities tools