Uniq
lastmod 2020-07-31created 2020-07-312 edits
Report or filter out repeated lines in a file.
Caveats
Use it on the assumption that the input is sorted.
Options
-
-c - Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-
-d - Only output lines that are repeated in the input.
-
-u - Only output lines that are not repeated in the input.
-
-f num - Ignore the first num fields in each input line when doing comparisons.
-
-i - Case insensitive comparison of lines.
-
-s chars - Ignore the first chars characters in each input line when doing comparisons.
Examples
-
sort yourfile.txt | uniq -d - Print only duplicated lines
-
sort yourfile.txt | uniq -u - Print only non-duplicated lines