UNIQ(1) General Commands Manual UNIQ(1)

uniq — report or filter out repeated lines in a file

uniq [−c | −d | −u] [−f fields] [−s chars] [input_file [output_file]]
Obsolescent Version:
uniq [−c | −d | −u] [−n] [+m] [input_file [output_file]]

The uniq utility shall read an input file comparing adjacent lines, and write one copy of each input line on the output. The second and succeeding copies of repeated adjacent input lines shall not be written. Repeated lines in the input shall not be detected if they are not adjacent.

The uniq utility shall conform to the utility argument syntax guidelines described in 2.10.2; the obsolescent version does not, as one of the options begins with + and the −m and +n options do not have option letters. The following options shall be supported by the implementation:

Precede each output line with a count of the number of times the line occurred in the input.

Suppress the writing of lines that are not repeated in the input.

fields

Ignore the first fields fields on each input line when doing comparisons, where fields shall be a positive decimal integer. A field is the maximal string matched by the basic regular expresssion: [[:blank:]]∗[ˆ[:blank:]]∗ If the fields option-argument specifies more fields than appear on an input line, a null string shall be used for comparison.

chars

Ignore the first chars characters when doing comparisons, where chars shall be a positive decimal integer. If specified in conjunction with the −f option, the first chars characters after the first fields fields shall be ignored. If the chars option-argument specifies more characters than remain on an input line, a null string shall be used for comparison.

Suppress the writing of lines that are repeated in the input.

(Obsolescent.) Equivalent to −f fields with fields set to n. +m (Obsolescent.) Equivalent to −s chars with chars set to m.

The following operands shall be supported by the implementation:

input_file

A pathname of the input file. If the input_file operand is not specified, or if the input_file is −, the standard input shall be used.

output_file

A pathname of the output file. If the output_file operand is not specified, the standard output shall be used. The results are unspecified if the file named by output_file is the file named by input_file.

The standard input shall be used only if no input_file operand is specified or if input_file is −. See Input Files.

The input file shall be a text file.

The following environment variables shall affect the execution of uniq:

This variable shall determine the locale to use for the locale categories when both LC_ALL and the corresponding environment variable (beginning with LC_ ) do not specify a locale. See 2.6.

This variable shall determine the locale to be used to override any values for locale categories specified by the settings of LANG or any environment variables beginning with LC_.

This variable shall determine the locale for the interpretation of sequences of bytes of text data as characters (e.g., single- versus multibyte characters in arguments and input files) and which characters constitute a <blank> in the current locale.

This variable shall determine the language in which messages should be written.

Default.

The standard output shall be used only if no output_file operand is specified. See Output Files.

Used only for diagnostic messages.

If the −c option is specified, the output file shall be empty or each line will be of the form: "%d %s", <number of duplicates>, <line> otherwise, the output file will be empty or each line will be of the form: "%s", <line>

None.

The uniq utility shall exit with one of the following values:

The utility executed successfully.

An error occurred.

Default.

Some historical implementations have limited lines to be 1080 bytes in length, which will not meet the implied limit. The sort utility (see sort(1) ) can be used to cause repeated lines to be adjacent in the input file. The following input file data (but flushed left) was used for a test series on uniq: #01 foo0 bar0 foo1 bar1 #02 bar0 foo1 bar1 foo1 #03 foo0 bar0 foo1 bar1 #04 #05 foo0 bar0 foo1 bar1 #06 foo0 bar0 foo1 bar1 #07 bar0 foo1 bar1 foo0 What follows is a series of test invocations of the uniq utility that use a mixture of uniq ’s options against the input file data. These tests verify the meaning of adjacent. The uniq utility views the input data as a sequence of strings delimited by \n. Accordingly, for the fieldsth member of the sequence, uniq interprets unique or repreated adjacent lines strictly relative to the fields+1th member. This first example tests the line counting option, comparing each line of the input file data starting from the second field: uniq −c −f 1 uniq_0I.t 1 #01 foo0 bar0 foo1 bar1 1 #02 bar0 foo1 bar1 foo0 1 #03 foo0 bar0 foo1 bar1 1 #04 2 #05 foo0 bar0 foo1 bar1 1 #07 bar0 foo1 bar1 foo0 The number 2, prefixing the fifth line of output, signifies that the uniq utility detected a pair of repeated lines. Given the input data, this can only be true when uniq is run using the −f 1 option (which causes uniq to ignore the first field on each input line). The second example tests the option to suppress unique lines, comparing each line of the input file data starting from the second field: uniq −d −f 1 uniq_0I.t #05 foo0 bar0 foo1 bar1 This test suppresses repeated lines, comparing each line of the input file data starting from the second field:

uniq −u −f 1 uniq_0I.t #01 foo0 bar0 foo1 bar1 #02 bar0 foo1 bar1 foo1 #03 foo0 bar0 foo1 bar1 #04 #07 bar0 foo1 bar1 foo0 This suppresses unique lines, comparing each line of the input file data starting from the third character: uniq −d −s 2 uniq_0I.t In the last example, the uniq utility found no input matching the above criteria.

The −f and −s options were added to replace the obsolescent −n and +m options so that uniq could meet the syntax guidelines in an upward-compatible way. The output specifications in Output Files do not show a terminating <newline> because they both specify <line>, which includes its own <newline> (because of the definition of line).

September 1991 posix.fail