Remove duplicate words
Drop repeated words across the whole text, keeping each first occurrence. Matching ignores case, but punctuation attached to a word makes it different.
Enter text to see output.
What to know
This operation works on words rather than lines. It splits the text at spaces, tabs and line breaks, keeps the first appearance of each word, and drops every later one, wherever it sits in the text. The cat and the dog and the bird turns into the cat and dog bird.
Words are compared after lowercasing, so The, the and THE are the same word and only the first form survives. The comparison uses the whole space-separated chunk, so hello, with a comma is a different word from hello, and both stay when they appear together.
The scan covers the entire input, not each line separately. A word that also occurred on line one is removed from line three, which can leave a line shorter or empty, and the leftover gap may put a space at the start or end of a line. Running trim whitespace afterwards tidies that up.
Line breaks are preserved, but any run of two or more spaces or tabs, anywhere in the text, is squeezed to one space. This helps with tag lists, search queries and pasted keyword strings, though it is the wrong tool for prose that should read naturally.
Choose a specific task 10
Related tools 4
How to use Remove duplicate words
- Paste a keyword string such as seo tool seo checker free tool into the Text input box.
- Check that the Output panel shows each word once, here seo tool checker free.
- If a word with a comma survives twice, remove the commas first with the remove punctuation operation, then rerun this one.
- Copy the Output panel, then press trim whitespace in the Operation menu if a stray space is left at a line edge.
Quick answers
Why does a repeated word with a comma survive?
The comparison treats the word and its attached punctuation as one chunk, so hello, and hello are different. Strip punctuation first to merge them.
Does it work line by line?
No, it tracks words across the whole text, so a word seen on line one is removed from every later line too.
Will it break my paragraph layout?
Line breaks stay, but all runs of two or more spaces or tabs become one space, so indentation is lost. A leftover gap can also leave a space at a line edge.
Is The the same as the here?
Yes. Words are lowercased before they are compared, so the first form you wrote, whether The or the, is the one that stays.