STANDARD INPUT

The standard input shall be used only if no file operands are specified, or if a file operand is −. See Input Files.

INPUT FILES

Input files to the awk program from any of the following sources:

• Any file operands or their equivalents, achieved by modifying the awk variables ARGV and ARGC

• Standard input in the absence of any file operands

• Arguments to the getline function

shall be text files. Whether the variable RS is set to a value other than <newline> or not, for these files, the implementation shall support records terminated with the specified separator up to LINE_MAX bytes and may support longer records.

If −f progfile is specified, the file(s) named by progfile shall be text file(s) containing an awk program.

ENVIRONMENT VARIABLES

The following environment variables shall affect the execution of awk:

LANG

This variable shall determine the locale to use for the locale categories when both LC_ALL and the corresponding environment variable (beginning with LC_ ) do not specify a locale. See 2.6.

LC_ALL

This variable shall determine the locale to be used to override any values for locale categories specified by the settings of LANG or any environment variables beginning with LC_.

LC_CTYPE

This variable shall determine the locale for the interpretation of sequences of bytes of text data as characters (e.g., single- versus multibyte characters in arguments and input files), the behavior of character classes within regular expressions, the identification of characters as letters, and the mapping of upper- and lowercase characters for the toupper and tolower functions.

LC_COLLATE

This variable shall determine the locale for the behavior of ranges, equivalence classes, and multicharacter collating elements within regular expressions and in comparisons of string values.

LC_MESSAGES

This variable shall determine the language in which messages should be written.

LC_NUMERIC

This variable shall determine the radix character used when interpreting numeric input, performing conversions between numeric and string values, and formatting numeric output.

PATH

This variable shall define the search path when looking for commands executed by system(expr), or input and output pipes. See 2.6. In addition, all environment variables shall be visible via the awk variable ENVIRON.

ASYNCHRONOUS EVENTS

Default.

STANDARD OUTPUT

The nature of the output files depends on the awk program.

STANDARD ERROR

Used only for diagnostic messages.

OUTPUT FILES

The nature of the output files depends on the awk program.

OVERALL PROGRAM STRUCTURE

An awk program is composed of pairs of the form: pattern { action } Either the pattern or the action (including the enclosing brace characters) can be omitted. A missing pattern shall match any record of input, and a missing action shall be equivalent to an action that writes the matched record of input to standard output. Execution of the awk program shall start by first executing the actions associated with all BEGIN patterns in the order they occur in the program. Then each file operand (or standard input if no files were specified) shall be processed in turn by reading data from the file until a record separator is seen (<newline> by default), splitting the current record into fields using the current value of FS according to the rules in REGULAR EXPRESSIONS , evaluating each pattern in the program in the order of occurrence, and executing the action associated with each pattern that matches the current record. The action for a matching pattern shall be executed before evaluating subsequent patterns. Last, the actions associated with all END patterns shall be executed in the order they occur in the program.

EXPRESSIONS

Expressions describe computations used in patterns and actions. In Table 4-1, valid expression operations are given in groups from highest precedence first to lowest precedence last, with equal-precedence operators grouped between horizontal lines. In expression evaluation, higher precedence operators shall be evaluated before lower precedence operators. In this table expr, expr1, expr2, and expr3 represent any expression, while lvalue represents any entity that can be assigned to (i.e., on the left side of an assignment operator). The precise syntax of expressions is given in the grammar in AWK GRAMMAR . Each expression shall have either a string value, a numeric value, or both. Except as stated for specific contexts, the value of an expression shall be implicitly converted to the type needed for the context in which it is used. A string value shall be converted to a numeric value by the equivalent of the following calls to functions defined by the C Standard {7}: setlocale( LC_NUMERIC, ""); numeric_value = atof(string_value); A numeric value that is exactly equal to the value of an integer (see 2.9.2.1) shall

Table 4-1 − awk Expressions in Decreasing Precedence

Semantic Type of Syntax Name Definition Result Assoc

( expr ) Grouping C Standard {7} type of expr n/a Field reference NUMERIC LEFT string n/a $expr

++ lvalue Pre-increment C Standard {7} numeric n/a −− lvalue Pre-decrement C Standard {7} numeric n/a lvalue ++ Post-increment C Standard {7} numeric n/a Post-decrement C Standard {7} numeric n/a lvalue −−

expr ˆ expr Exponentiation

NUMERIC RIGHT

! expr Logical not C Standard {7} numeric n/a + expr Unary plus C Standard {7} numeric n/a − expr Unary minus C Standard {7} numeric n/a 186 expr ∗ expr Multiplication C Standard {7} numeric left

expr / expr Division C Standard {7} numeric left expr % expr Modulus

NUMERIC LEFT

expr + expr Addition C Standard {7} numeric left expr − expr Subtraction C Standard {7} numeric left expr expr String concatenation NUMERIC LEFT
string left

expr < expr Less than NUMERIC LEFT
numeric none expr <= expr Less than or equal to NUMERIC LEFT
numeric none

expr != expr Not equal to NUMERIC LEFT
numeric none expr == expr Equal to NUMERIC LEFT numeric none

expr > expr Greater than NUMERIC LEFT
numeric none expr >= expr Greater than or equal to NUMERIC LEFT
numeric none

expr ∼ expr ERE match REGULAR EXPRESSIONS
numeric none expr !∼ expr ERE nonmatch REGULAR EXPRESSIONS
numeric none

expr in array Array membership NUMERIC LEFT
numeric left ( index ) in Multidimension array NUMERIC LEFT
numeric left

array membership expr && expr Logical AND C Standard {7} numeric left

expr | | expr Logical OR C Standard {7} numeric left expr1 ? expr2 Conditional expression C Standard {7} type of selected right : expr3 expr2 or expr3 206

lvalue ˆ= expr Exponentiation NUMERIC LEFT
numeric right assignment lvalue %= expr Modulus assignment NUMERIC LEFT
numeric right

lvalue ∗= expr Multiplication C Standard {7} numeric right assignment lvalue /= expr Division assignment C Standard {7} numeric right

lvalue += expr Addition assignment C Standard {7} numeric right lvalue −= expr Subtraction assignment C Standard {7} numeric right lvalue = expr Assignment C Standard {7} type of expr right

be converted to a string by the equivalent of a call to the sprintf function (see FUNCTIONS ) with the string "%d" as the fmt argument and the numeric value being converted as the first and only expr argument. Any other numeric value shall be converted to a string by the equivalent of a call to the sprintf function with the value of the variable CONVFMT as the fmt argument and the numeric value being converted as the first and only expr argument. The result of the conversion is unspecified if the value of CONVFMT is not a floating-point format specification. This standard specifies no explicit conversions between numbers and strings. An application can force an expression to be treated as a number by adding zero to it, or can force it to be treated as a string by concatenating the null string ("") to it. A string value shall be considered to be a numeric string in the following case:

1. Any leading and trailing <blank>s shall be ignored.
2. If the first unignored character is a + or −, it shall be ignored.
3. If the remaining unignored characters would be lexically recognized as a NUMBER token (as described by the lexical conventions in AWK GRAMMAR ), the string shall be considered a numeric string.

If a − character is ignored in the above steps, the numeric value of the numeric string shall be the negation of the numeric value of the recognized NUMBER token. Otherwise the numeric value of the numeric string shall be the numeric value of the recognized NUMBER token. Whether or not a string is a numeric string shall be relevant only in contexts where that term is used in this clause. When an expression is used in a Boolean context (the first subexpression of a conditional expression, an expression operated on by logical NOT, logical AND, or logical OR, the second expression of a for statement, the expression of an if statement, or the expression of a while statement), if it has a numeric value, a value of zero shall be treated as false and any other value shall be treated as true. Otherwise, a string value of the null string shall be treated as false and any other value shall be treated as true. All arithmetic shall follow the semantics of floating point arithmetic as specified by the C Standard {7}; see 2.9.2. The value of the expression expr1 ˆ expr2 shall be equivalent to the value returned by the C Standard {7} function call pow(expr1, expr2) The expression lvalue ˆ= expr shall be equivalent to the C Standard {7} expression lvalue = pow(lvalue, expr) except that lvalue shall be evaluated only once. The value of the expression

expr1 % expr2 shall be equivalent to the value returned by the C Standard {7} function call fmod(expr1, expr2) The expression lvalue %= expr shall be equivalent to the C Standard {7} expression lvalue = fmod(lvalue, expr) except that lvalue shall be evaluated only once. Variables and fields shall be set by the assignment statement: lvalue = expression and the type of expression shall determine the resulting variable type. The assignment includes the arithmetic assignments (+=, -=, ∗=, /=, %=, ˆ=, ++, --) all of which produce a numeric result. The left-hand side of an assignment and the target of increment and decrement operators can be one of a variable, an array with index, or a field selector. The awk language shall supply arrays that are used for storing numbers or strings. Arrays need not be declared. They shall initially be empty, and their sizes shall change dynamically. The subscripts, or element identifiers, are strings, providing a type of associative array capability. An array name followed by a subscript within square brackets can be used as an lvalue and thus as an expression, as described in the grammar (see AWK GRAMMAR ). Unsubscripted array names can be used in only the following contexts:

• A parameter in a function definition or function call.
• The NAME token following any use of the keyword in as specified in the grammar (see AWK GRAMMAR ). If the name used in this context is not an array name, the behavior is undefined.

A valid array index shall consist of one or more comma-separated expressions, similar to the way in which multidimensional arrays are indexed in some programming languages. Because awk arrays are really one dimensional, such a comma-separated list shall be converted to a single string by concatenating the string values of the separate expressions, each separated from the other by the value of the SUBSEP variable. Thus, the following two index operations shall be equivalent: var[expr1, expr2, . . . , exprn] var[expr1 SUBSEP expr2 SUBSEP . . .SUBSEP exprn] A multidimensioned index used with the in operator shall be parenthesized. The in operator, which tests for the existence of a particular array element, shall not cause that element to exist. Any other reference to a nonexistent array element shall automatically create it.

Comparisons (with the <, <=, !=, ==, >, and >= operators) shall be made numerically if both operands are numeric or if one is numeric and the other has a string value that is a numeric string. Otherwise, operands shall be converted to strings as required and a string comparison shall be made using the locale-specific collation sequence. The value of the comparison expression shall be 1 if the relation is true, or 0 if the relation is false.

VARIABLES AND SPECIAL VARIABLES

Variables can be used in an awk program by referencing them. With the exception of function parameters (see FUNCTIONS ), they are not explicitly declared. Uninitialized scalar variables and array elements have both a numeric value of zero and a string value of the empty string. Field variables shall be designated by a $ followed by a number or numerical expression. The effect of the field number expression evaluating to anything other than a nonnegative integer is unspecified; uninitialized variables or string values need not be converted to numeric values in this context. New field variables can be created by assigning a value to them. References to nonexistent fields (i.e., fields after $NF), shall produce the null string. However, assigning to a nonexistent field [e.g., $(NF+2) = 5] shall increase the value of NF, create any intervening fields with the null string as their values, and cause the value of $0 to be recomputed, with the fields being separated by the value of OFS. Each field variable shall have a string value when created. If the string, with any occurrence of the decimal-point character from the current locale changed to a <period>, would be considered a numeric string (see NUMERIC LEFT ), the field variable shall also have the numeric value of the numeric string. The implementation shall support the following other special variables that are set by awk: ARGC The number of elements in the ARGV array. ARGV An array of command line arguments, excluding options and the program argument, numbered from zero to ARGC −1. The arguments in ARGV can be modified or added to; ARGC can be altered. As each input file ends, awk shall treat the next nonnull element of ARGV, up through the current value of ARGC −1, as the name of the next input file. Thus, setting an element of ARGV to null means that it shall not be treated as an input file. The name ’-’ shall indicate the standard input. If an argument matches the format of an assignment operand, this argument shall be treated as an assignment rather than a file argument. CONVFMT The printf format for converting numbers to strings (except for output statements, where OFMT is used); "%.6g" by default. ENVIRON The variable ENVIRON is an array representing the value of the environment, as described in POSIX. 1 {8} 2.7. The indices of the array shall be strings consisting of the names of the environment variables, and the value of each array element shall be a string

consisting of the value of that variable. If the value of an environment variable is considered a numeric string (see NUMERIC LEFT ), the array element shall also have its numeric value. In all cases where the behavior of awk is affected by environment variables [including the environment of any command(s) that awk executes via the system function or via pipeline redirections with the print statement, the printf statement, or the getline function], the environment used shall be the environment at the time awk began executing; it is implementation defined whether any modification of ENVIRON affects this environment.

FILENAME A pathname of the current input file. Inside a BEGIN action the value is undefined. Inside an END action the value is the name of the last input file processed. FNR The ordinal number of the current record in the current file. Inside a BEGIN action the value is zero. Inside an END action the value is the number of the last record processed in the last file processed. FS Input field separator regular expression; <space> by default. NF The number of fields in the current record. Inside a BEGIN action, the use of NF is undefined unless a getline function without a var argument is executed previously. Inside an END action, NF shall retain the value it had for the last record read, unless a subsequent, redirected, getline function without a var argument is performed prior to entering the END action. NR The ordinal number of the current record from the start of input. Inside a BEGIN action the value is zero. Inside an END action the value is the number of the last record processed. OFMT The printf format for converting numbers to strings in output statements (see OUTPUT STATEMENTS ); "%.6g" by default. The result of the conversion is unspecified if the value of OFMT is not a floatingpoint format specification.

OFS The print statement output field separation; <space> by default. ORS The print statement output record separator; <newline> by default. RLENGTH The length of the string matched by the match function. RS The first character of the string value of RS is the input record separator; <newline> by default. If RS contains more than one character, the results are unspecified. If RS is null, then records are separated by sequences of one or more blank lines, leading or trailing blank lines do not result in empty records at the beginning or end of the input, and <newline> is always a field separator, no matter what the value of FS is.

RSTART The starting position of the string matched by the match function, numbering from 1. This is always equivalent to the return value of the match function. SUBSEP The subscript separator string for multidimensional arrays; the default value is implementation defined.

REGULAR EXPRESSIONS

The awk utility shall make use of the extended regular expression notation (see 2.8.4) except that it shall allow the use of C-language conventions for escaping special characters within the EREs, as specified in Table 2-15 and Table 4-2; these escape sequences shall be recognized both inside and outside bracket expressions. Note that records need not be separated by <newline>s and string constants can contain <newline>s, so even the \n sequence is valid in awk EREs. Using a slash character within the regular expression requires the escaping shown in Table 4-2.

A regular expression can be matched against a specific field or string by using one of the two regular expression matching operators, ∼ and !∼. These operators shall interpret their right-hand operand as a regular expression and their left-hand operand as a string. If the regular expression matches the string, the ∼ expression shall evaluate to a value of 1, and the !∼ expression shall evaluate to a value of 0. (The regular expression matching operation is as defined in 2.8.1.2, where a match occurs on any part of the string unless the regular expression is limited with the circumflex or dollar-sign special characters.) If the regular expression does not match the string, the ∼ expression shall evaluate to a value of 0, and the !∼ expression shall evaluate to a value of 1. If the right-hand operand is any expression other than the lexical token ERE, the string value of the expression shall be interpreted as an extended regular expression, including the escape conventions described above. Note that these same escape conventions also shall be applied in the determining the value of a string literal (the lexical token STRING ), and thus shall be applied a second time when a string literal is used in this context. When an ERE token appears as an expression in any context other than as the right-hand of the ∼ or !∼ operator or as one of the built-in function arguments described below, the value of the resulting expression shall be the equivalent of $0 ∼ /ere/ The ERE argument to the gsub, match, sub functions, and the fs argument to the split function (see FUNCTIONS ) shall be interpreted as extended regular expressions. These can be either ERE tokens or arbitrary expressions, and shall be interpreted in the same manner as the right-hand side of the ∼ or !∼ operator. An extended regular expression can be used to separate fields by using the −F ERE option or by assigning a string containing the expression to the built-in variable FS. The default value of the FS variable shall be a single <space> character. The following describes FS behavior:

1. If FS is a single character: (a) If FS is <space>, skip leading and trailing <blank>s; fields shall be delimited by sets of one or more <blank>s. (b) Otherwise, if FS is any other character c, fields shall be delimited by each single occurrence of c.
2. Otherwise, the string value of FS shall be considered to be an extended regular expression. Each occurrence of a sequence matching the extended regular expression shall delimit fields.

Except in the gsub, match, split, and sub built-in functions, regular expression matching shall be based on input records; i.e., record separator characters (the first character of the value of the variable RS, <newline> by default) cannot be embedded in the expression, and no expression shall match the record separator character. If the record separator is not <newline>, <newline> characters embedded in the expression can be matched. In those four built-in functions, regular expression matching shall be based on text strings; i.e., any character (including <newline> and the record separator) can be embedded in the pattern and an appropriate pattern shall match any character. However, in all awk regular expression matching, the use of one or more NUL characters in the pattern, input record, or text string produces undefined results.

PATTERNS

A pattern is any valid expression, a range specified by two expressions separated by comma, or one of the two special patterns BEGIN or END.

SPECIAL PATTERNS

The awk utility shall recognize two special patterns, BEGIN and END. Each BEGIN pattern shall be matched once and its associated action executed before the first record of input is read [except possibly by use of the getline function (see FUNCTIONS ) in a prior BEGIN action] and before command line assignment is done. Each END pattern shall be matched once and its associated action executed after the last record of input has been read. These two patterns shall have associated actions. BEGIN and END shall not combine with other patterns. Multiple BEGIN and END patterns shall be allowed. The actions associated with the BEGIN patterns shall be executed in the order specified in the program, as are the END actions. An END pattern can precede a BEGIN pattern in a program. If an awk program consists of only actions with the pattern BEGIN, and the BEGIN action contains no getline function, awk shall exit without reading its input when the last statement in the last BEGIN action is executed. If an awk program consists of only actions with the pattern END or only actions with the patterns BEGIN and END, the input shall be read before the statements in the END action(s) are executed.

EXPRESSION PATTERNS

An expression pattern shall be evaluated as if it were an expression in a Boolean context. If the result is true, the pattern shall be considered to match, and the associated action (if any) shall be executed. If the result is false, the action shall not be executed.

PATTERN RANGES

A pattern range consists of two expressions separated by a comma; in this case, the action shall be performed for all records between a match of the first expression and the following match of the second expression, inclusive. At this point, the pattern range can be repeated starting at input records subsequent to the end of the matched range.

ACTIONS

An action is a sequence of statements as shown in the grammar in AWK GRAMMAR . Any single statement can be replaced by a statement list enclosed in braces. The statements in a statement list shall be separated by <newline>s or semicolons, and shall be executed sequentially in the order that they appear. The expression acting as the conditional in an if statement shall be evaluated and if it is nonzero or nonnull, the following statement shall be executed; otherwise, if else is present, the statement following the else shall be executed. The if, while, do . . . while, for, break, and continue statements are based on the C Standard {7} (see 2.9.2), except that the Boolean expressions shall be treated as described in NUMERIC LEFT , and except in the case of for (variable in array) which shall iterate, assigning each index of array to variable in an unspecified order. The results of adding new elements to array within such a for loop are undefined. If a break or continue statement occurs outside of a loop, the behavior is undefined. The delete statement shall remove an individual array element. Thus, the following code shall delete an entire array: for (index in array) delete array[index] The next statement shall cause all further processing of the current input record to be abandoned. The behavior is undefined if a next statement appears or is invoked in a BEGIN or END action. The exit statement shall invoke all END actions in the order in which they occur in the program source and then terminate the program without reading further input. An exit statement inside an END action shall terminate the program without further execution of END actions. If an expression is specified in an exit statement, its numeric value shall be the exit status of awk, unless subsequent errors are encountered or a subsequent exit statement with an expression is

executed.

OUTPUT STATEMENTS

Both print and printf statements shall write to standard output by default. The output shall be written to the location specified by output_redirection if one is supplied, as follows: > expression >> expression

In all cases, the expression shall be evaluated to produce a string that is used as a full pathname to write into (for > or >>) or as a command to be executed (for |). Using the first two forms, if the file of that name is not currently open, it shall be opened, creating it if necessary, and using the first form, truncating the file. The output then shall be appended to the file. As long as the file remains open, subsequent calls in which expression evaluates to the same string value simply shall append output to the file. The file remains open until the close function (see FUNCTIONS ). is called with an expression that evaluates to the same string value. The third form shall write output onto a stream piped to the input of a command. The stream shall be created if no stream is currently open with the value of expression as its command name. The stream created shall be equivalent to one created by a call to the popen() function (see B.3.2) with the value of expression as the command argument and a value of "w" as the mode argument. As long as the stream remains open, subsequent calls in which expression evaluates to the same string value shall write output to the existing stream. The stream shall remain open until the close function (see FUNCTIONS ) is called with an expression that evaluates to the same string value. At that time, the stream shall be closed as if by a call to the pclose() function (see B.3.2). As described in detail by the grammar in AWK GRAMMAR , these output statements shall take a comma-separated list of expressions referred in the grammar by the nonterminal symbols expr_list, print_expr_list, or print_expr_list_opt. This list is referred to here as the expression list, and each member is referred to as an expression argument. The print statement shall write the value of each expression argument onto the indicated output stream separated by the current output field separator (see variable OFS above), and terminated by the output record separator (see variable ORS above). All expression arguments shall be taken as strings, being converted if necessary; this conversion shall be as described in NUMERIC LEFT , with the exception that the printf format in OFMT shall be used instead of the value in CONVFMT. An empty expression list shall stand for the whole input record ($0). The printf statement shall produce output based on a notation similar to the File Format Notation used to describe file formats in this standard (see 2.12). Output shall be produced as specified with the first expression argument as the string <format> and subsequent expression arguments as the strings <arg1> through <argn>, with the following exceptions:

1. The format shall be an actual character string rather than a graphical representation. Therefore, it cannot contain empty character positions. The <space> character in the format string, in any context other than a flag of a conversion specification, shall be treated as an ordinary character that is copied to the output.
2. If the character set contains a ∆ character and that character appears in the format string, it shall be treated as an ordinary character that is copied to the output.
3. The escape sequences beginning with a backslash character shall be treated as sequences of ordinary characters that are copied to the output. (Note that these same sequences shall be interpreted lexically by awk when they appear in literal strings, but they shall not be treated specially by the printf statement).
4. A field width or precision can be specified as the ∗ character instead of a digit string. In this case the next argument from the expression list shall be fetched and its numeric value taken as the field width or precision.
5. The implementation shall not precede or follow output from the d or u conversion specifications with <blank>s not specified by the format string.
6. The implementation shall not precede output from the o conversion specification with leading zeroes not specified by the format string.
7. For the c conversion specification: if the argument has a numeric value, the character whose encoding is that value shall be output. If the value is zero or is not the encoding of any character in the character set, the behavior is undefined. If the argument does not have a numeric value, the first character of the string value shall be output; if the string does not contain any characters the behavior is undefined.
8. For each conversion specification that consumes an argument, the next expression argument shall be evaluated. With the exception of the c conversion, the value shall be converted (according to the rules specified in NUMERIC LEFT ) to the appropriate type for the conversion specification.
9. If there are insufficient expression arguments to satisfy all the conversion specifications in the format string, the behavior is undefined.
10. If any character sequence in the format string begins with a % character, but does not form a valid conversion specification, the behavior is unspecified.

Both print and printf can output at least LINE_MAX bytes.

FUNCTIONS

The awk language has a variety of built-in functions: arithmetic, string, input/output, and general.

ARITHMETIC FUNCTIONS

The arithmetic functions, except for int, shall be based on the C Standard {7}; see 2.9.2. The behavior is undefined in cases where the C Standard {7} specifies that an error be returned or that the behavior is undefined. atan2(y,x) Return arctangent of y/x. cos(x) Return cosine of x, where x is in radians. sin(x) Return sine of x, where x is in radians. exp(x) Return the exponential function of x. log(x) Return the natural logarithm of x. sqrt(x) Return the square root of x. int(x) Truncate its argument to an integer. It shall be truncated toward 0 when x > 0. rand() Return a random number n, such that 0 ≤ n < 1. srand([expr]) Set the seed value for rand to expr or use the time of day if expr is omitted. The previous seed value shall be returned.

STRING FUNCTIONS

The string functions are: gsub(ere, repl[, in]) Behave like sub (see below), except that it shall replace all occurrences of the regular expression (like the ed utility global substitute) in $0 or in the in argument, when specified. index(s, t) Return the position, in characters, numbering from 1, in string s where string t first occurs, or zero if it does not occur at all. length([s]) Return the length, in characters, of its argument taken as a string, or of the whole record, $0, if there is no argument. match(s, ere) Return the position, in characters, numbering from 1, in string s where the extended regular expression ERE occurs, or zero if it does not occur at all. RSTART shall be set to the starting position (which is the same as the returned value), zero if no match is found; RLENGTH shall be set to the length of the matched string, −1 if no match is found. split(s, a[, fs]) Split the string s into array elements a[1], a[2], . . . , a[n], and returns n. The separation shall be done with the extended regular expression fs or with the field separator

FS if fs is not given. Each array element shall have a string value when created. If the string assigned to any array element, with any occurrence of the decimal-point character from the current locale changed to a <period>, would be considered a numeric string (see NUMERIC LEFT ), the array element shall also have the numeric value of the numeric string. The effect of a null string as the value of fs is unspecified. sprintf(fmt, expr, expr, . . . ) Format the expressions according to the printf format given by fmt and return the resulting string. sub(ere, repl[, in]) Substitute the string repl in place of the first instance of the extended regular expression ERE in string in and return the number of substitutions. An ampersand (&) appearing in the string repl shall be replaced by the string from in that matches the regular expression. An ampersand preceded by a backslash within repl shall be interpreted as a literal ampersand character. If in is specified and it is not an lvalue (see NUMERIC LEFT ), the behavior is undefined. If in is omitted, awk shall substitute in the current record ($0). substr(s, m[, n]) Return the at most n-character substring of s that begins at position m, numbering from 1. If n is missing, the length of the substring shall be limited by the length of the string s. tolower(s) Return a string based on the string s. Each character in s that is an uppercase letter specified to have a tolower mapping by the LC_CTYPE category of the current locale shall be replaced in the returned string by the lowercase letter specified by the mapping. Other characters in s shall be unchanged in the returned string. toupper(s) Return a string based on the string s. Each character in s that is a lowercase letter specified to have a toupper mapping by the LC_CTYPE category of the current locale shall be replaced in the returned string by the uppercase letter specified by the mapping. Other characters in s shall be unchanged in the returned string. All of the preceding functions that take ERE as a parameter expect a pattern or a string valued expression that is a regular expression as defined in REGULAR EXPRESSIONS .

INPUT/OUTPUT AND GENERAL FUNCTIONS

The input/output and general functions are: close(expression) Close the file or pipe opened by a print or printf statement or a call to getline with the same string-valued expression. The limit on the number of open expression arguments is implementation defined. If the close was successful, the function shall return zero; otherwise, it shall return nonzero. expression | getline [var] Read a record of input from a stream piped from the output of a command. The stream shall be created if no stream is currently open with the value of expression as its command name. The stream created shall be equivalent to one created by a call to the popen() function with the value of expression as the command argument and a value of "r" as the mode argument. As long as the stream remains open, subsequent calls in which expression evaluates to the same string value shall read subsequent records from the file. The stream shall remain open until the close function is called with an expression that evaluates to the same string value. At that time, the stream shall be closed as if by a call to the pclose() function. If var is missing, $0 and NF shall be set; otherwise, var shall be set. getline Set $0 to the next input record from the current input file. This form of getline shall set the NF, NR, and FNR variables. getline var Set variable var to the next input record from the current input file. This form of getline shall set the FNR and NR variables. getline [var] < expression Read the next record of input from a named file. The expression shall be evaluated to produce a string that is used as a full pathname. If the file of that name is not currently open, it shall be opened. As long as the stream remains open, subsequent calls in which expression evaluates to the same string value shall read subsequent records from the file. The file shall remain open until the close function is called with an expression that evaluates to the same string value. If var is missing, $0 and NF shall be set; otherwise, var shall be set. system(expression) Execute the command given by expression in a manner equivalent to the system() function [see B.3.1] and return

the exit status of the command. All forms of getline shall return 1 for successful input, zero for end of file, and −1 for an error.

USER-DEFINED FUNCTIONS

The awk language also shall provide user-defined functions. Such functions can be defined as: function name(args,. . . ) { statements } A function can be referred to anywhere in an awk program; in particular, its use can precede its definition. The scope of a function shall be global. Function arguments can be either scalars or arrays; the behavior is undefined if an array name is passed as an argument that the function uses as a scalar, or if a scalar expression is passed as an argument that the function uses as an array. Function arguments shall be passed by value if scalar and by reference if array name. Argument names shall be local to the function; all other variable names shall be global. The same name shall not be used as both an argument name and as the name of a function or a special awk variable. The same name shall not be used both as a variable name with global scope and as the name of a function. The same name shall not be used within the same scope both as a scalar variable and as an array. The number of parameters in the function definition need not match the number of parameters in the function call. Excess formal parameters can be used as local variables. If fewer arguments are supplied in a function call than are in the function definition, the extra parameters that are used in the function body as scalars shall be initialized with a string value of the null string and a numeric value of zero, and the extra parameters that are used in the function body as arrays shall be initialized as empty arrays. If more arguments are supplied in a function call than are in the function definition, the behavior is undefined. When invoking a function, no white space can be placed between the function name and the opening parenthesis. The implementation shall permit function calls to be nested, and for recursive calls to be made upon functions. Upon return from any nested or recursive function call, the values of all of the calling function’s parameters shall be unchanged, except for array parameters passed by reference. The return statement can be used to return a value. If a return statement appears outside of a function definition, the behavior is undefined. In the function definition, <newline>s shall be optional before the opening brace and after the closing brace. Function definitions can appear anywhere in the program where a pattern-action pair is allowed.

AWK GRAMMAR

The grammar in this subclause and the lexical conventions in the following subclause shall together describe the syntax for awk programs. The general conventions for this style of grammar are described in 2.1.2. A valid program can be

represented as the nonterminal symbol program in the grammar. Any discrepancies found between this grammar and other descriptions in this clause shall be resolved in favor of this grammar.

%token NAME NUMBER STRING ERE NEWLINE %token FUNC_NAME /∗ name followed by ’(’ without white space ∗/ /∗ Keywords ∗/ %token Begin End /∗ ’ BEGIN ’ ’ END ’ ∗/ %token Break Continue Delete Do Else /∗ ’break’ ’continue’ ’delete’ ’do’ ’else’ ∗/ %token Exit For Function If In /∗ ’exit’ ’for’ ’function’ ’if’ ’in’ ∗/ %token Next Print Printf Return While /∗ ’next’ ’print’ ’printf’ ’return’ ’while’ ∗/ /∗ Reserved function names ∗/ %token BUILTIN_FUNC_NAME /∗ one token for the following: ∗ atan2 cos sin exp log sqrt int rand srand ∗ gsub index length match split sprintf sub substr ∗ tolower toupper close system ∗/ %token GETLINE /∗ Syntactically different from other built-ins ∗/ /∗ Two-character tokens ∗/ %token ADD_ASSIGN SUB_ASSIGN MUL_ASSIGN DIV_ASSIGN MOD_ASSIGN POW_ASSIGN /∗ ’+=’ ’-=’ ’∗=’ ’/=’ ’%=’ ’ˆ=’ ∗/ %token OR AND NO_MATCH EQ LE GE NE INCR DECR APPEND /∗ ’| |’ ’&&’ ’!∼’ ’==’ ’<=’ ’>=’ ’!=’ ’++’ ’--’ ’>>’ ∗/ /∗ One-character tokens ∗/ %token ’{’ ’}’ ’(’ ’)’ ’[’ ’]’ ’,’ ’;’ %token ’+’ ’-’ ’∗’ ’%’ ’ˆ’ ’!’ ’>’ ’<’ ’|’ ’?’ ’:’ ’∼’ ’$’ ’=’ %start program %%

program: item_list

; item_list: newline_opt

; actionless_item_list: item_list pattern terminator

; item: pattern action

; param_list_opt: /∗ empty ∗/

; param_list: NAME

; pattern: Begin

; action: ’{’ newline_opt ’}’

; terminator: ’;’

; terminated_statement_list: terminated_statement

; unterminated_statement_list: unterminated_statement

; terminated_statement: action newline_opt

Else newline_opt terminated_statement

newline_opt terminated_statement

; unterminated_statement: terminatable_statement

Else newline_opt unterminated_statement

newline_opt unterminated_statement

; terminatable_statement: simple_statement

; simple_statement_opt: /∗ empty ∗/

; simple_statement: Delete NAME ’[’ expr_list ’]’

; print_statement: simple_print_statement

; simple_print_statement: Print print_expr_list_opt

; output_redirection: ’>’ expr

; expr_list_opt: /∗ empty ∗/

; expr_list: expr

; multiple_expr_list: expr ’,’ newline_opt expr

; expr_opt: /∗ empty ∗/

; expr: unary_expr

; unary_expr: ’+’ expr

; non_unary_expr: ’(’ expr ’)’

; print_expr_list_opt: /∗ empty ∗/

; print_expr_list: print_expr

; print_expr: unary_print_expr

; unary_print_expr: ’+’ print_expr

; non_unary_print_expr: ’(’ expr ’)’

; lvalue: NAME

; non_unary_input_function: simple_get

; unary_input_function: unary_expr ’|’ simple_get ; simple_get: GETLINE

; newline_opt: /∗ empty ∗/

;

This grammar has several ambiguities that shall be resolved as follows:

• Operator precedence and associativity shall be as described in Table 4-1.
• In case of ambiguity, an else shall be associated with the most immediately preceding if that would satisfy the grammar.

AWK LEXICAL CONVENTIONS

The lexical conventions for awk programs, with respect to the preceding grammar, shall be as follows:

1. Except as noted, awk shall recognize the longest possible token or delimiter beginning at a given point.
2. A comment shall consist of any characters beginning with the number sign character and terminated by, but excluding the next occurrence of, a <newline> character. Comments shall have no effect, except to delimit lexical tokens.
3. The character <newline> shall be recognized as the token NEWLINE.
4. A backslash character immediately followed by a <newline> character shall have no effect.

1. The token STRING shall represent a string constant. A string constant shall begin with the character ". Within a string constant, a backslash character shall be considered to begin an escape sequence as specified in Table 2-15 (see 2.12). In addition, the escape sequences in Table 4-2 shall be recognized. A <newline> character shall not occur within a string constant. A string constant shall be terminated by the first unescaped occurrence of the character " after the one that begins the string constant. The value of the string shall be the sequence of all unescaped characters and values of escape sequences between, but not including, the two delimiting " characters.
2. The token ERE represents an extended regular expression constant. An ERE constant shall begin with the slash character. Within an ERE constant, a <backslash> character shall be considered to begin an escape sequence as specified in Table 2-15 (see 2.12). In addition, the escape sequences in Table 4-2 shall be recognized. A <newline> character shall not occur within an ERE constant. An ERE constant shall be terminated by the first unescaped occurrence of the slash character after the one that begins the string constant. The extended regular expression represented by the ERE constant shall be the sequence of all unescaped characters and values of escape sequences between, but not including, the two delimiting slash characters.
3. A <blank> shall have no effect, except to delimit lexical tokens or within STRING or ERE tokens.
4. The token NUMBER shall represent a numeric constant. Its form and numeric value shall be equivalent to the either of the tokens floatingconstant or integer-constant as specified by the C Standard {7}, with the following exceptions:

Table 4-2 − awk Escape Sequences

Escape Description Meaning Sequence

\" <backslash> <quotation-mark> <quotation-mark> character

\/ <backslash> <slash> <slash> character

\ddd <backslash> followed by the long- The character whose encoding is est sequence of one, two, or three represented by the one-, two-, or threeoctal-digit characters (01234567). If digit octal integer. If the size of a byte on all of the digits are 0, (i.e., represen- the system is greater than nine bits, the tation of the NUL character), the valid escape sequence used to represent a behavior is undefined. byte is implementation defined. Multibyte characters require multiple, concatenated escape sequences of this type, including the leading \ for each byte.

\c <backslash> followed by any char- Undefined acter not described in this table or in Table 2-15

(a) An integer constant cannot begin with 0x or include the hexadecimal digits a, b, c, d, e, f, A, B, C, D, E, or F. (b) The value of an integer constant beginning with 0 shall be taken in decimal rather than octal. (c) An integer constant cannot include a suffix (u, U, l, or L). (d) A floating constant cannot include a suffix (f, F, l, or L). If the value is too large or too small to be representable (see 2.9.2.1), the behavior is undefined.

1. A sequence of underscores, digits, and alphabetics from the portable character set (see 2.4), beginning with an underscore or alphabetic, shall be considered a word.
2. The following words are keywords that shall be recognized as individual tokens; the name of the token is the same as the keyword:

BEGIN delete for in printf END do function next return break else getline print while continue exit if

1. The following words are names of built-in functions and shall be recognized as the token BUILTIN_FUNC_NAME:

atan2 index match sprintf substr close int rand sqrt system

cos length sin srand tolower exp log split sub toupper gsub

The above-listed keywords and names of built-in functions are considered reserved words.

1. The token NAME shall consist of a word that is not a keyword or a name of a built-in function and is not followed immediately (without any delimiters) by the ( character.
2. The token FUNC_NAME shall consist of a word that is not a keyword or a name of a built-in function, followed immediately (without any delimiters) by the ( character. The ( character shall not be included as part of the token.
3. The following two-character sequences shall be recognized as the named tokens:

Token Name Sequence Token

Name Sequence ADD_ASSIGN += NO_MATCH !∼ SUB_ASSIGN -= EQ == MUL_ASSIGN ∗= LE <= DIV_ASSIGN /= GE >= MOD_ASSIGN %= NE != POW_ASSIGN ˆ= INCR ++ OR || DECR −− AND && APPEND >>

1. The following single characters shall be recognized as tokens whose names are the character: <newline> { } ( ) [ ] , ; + − ∗ % ˆ ! > < | ? : ∼ $ =

There is a lexical ambiguity between the token ERE and the tokens / and DIV_ASSIGN. When an input sequence begins with a slash character in any syntactic context where the token / or DIV_ASSIGN could appear as the next token in a valid program, the longer of those two tokens that can be recognized shall be recognized. In any other syntactic context where the token ERE could appear as the next token in a valid program, the token ERE shall be recognized.

EXAMPLES

The awk program specified in the command line is most easily specified within single-quotes (e.g., ’program’) for applications using sh, because awk programs commonly contain characters that are special to the shell, including doublequotes. In the cases where an awk program contains single-quote characters, it is usually easiest to specify most of the program as strings within single-quotes concatenated by the shell with quoted single-quote characters. For example, awk ’/’\’’/ { print "quote:", $0 }’ prints all lines from the standard input containing a single-quote character, prefixed with quote:. The following are examples of simple awk programs:

1. Write to the standard output all input lines for which field 3 is greater than 5. $3 > 5
2. Write every tenth line. (NR % 10) == 0
3. Write any line with a substring matching the regular expression. /(G | D)(2[0-9][[:alpha:]]∗)/
4. Write any line in which the second field matches the regular expression and the fourth field does not. $2 ∼ /xyz/ && $4 !∼ /xyz/
5. Write any line in which the second field contains a backslash. $2 ∼ /\\/
6. Write any line in which the second field contains a backslash. Note that backslash escapes are interpreted twice, once in lexical processing of the string and once in processing the regular expression. $2 ∼ "\\\\"
7. Write the second to the last and the last field in each line. Separate the fields by a colon.

{ OFS =":";print $(NF-1), $NF}

1. Write the line number and number of fields in each line. The three strings representing the line number, the colon and the number of fields are concatenated and that string is written to standard output. {print NR ":" NF}
2. Write lines longer than 72 characters. {length($0) > 72}
3. Write first two fields in opposite order separated by the OFS: { print $2, $1 }
4. Same, with input fields separated by comma and/or <space>s and <tab>s: BEGIN { FS = ",[ \t]∗|[ \t]+" } { print $2, $1 }
5. Add up first column, print sum and average. {s += $1 } END {print "sum is ", s, " average is", s/NR}
6. Write fields in reverse order, one per line (many lines out for each line in): { for (i = NF; i > 0; --i) print $i }
7. Write all lines between occurrences of the strings start and stop: /start/, /stop/
8. Write all lines whose first field is different from the previous one: $1 != prev { print; prev = $1 }
9. Simulate echo: BEGIN { for (i = 1; i < ARGC; ++i) printf "%s%s", ARGV [i], i== ARGC -1?"\n":"" }
10. Write the path prefixes contained in the PATH environment variable, one per line: BEGIN { n = split ( ENVIRON [" PATH "], path, ":") for (i = 1; i <= n; ++i) print path[i] }
11. If there is a file named ‘‘input’’ containing page headers of the form: Page # and a file named ‘‘program’’ that contains:

/Page/{ $2 = n++; } { print } then the command line: awk −f program n=5 input will print the file ‘‘input,’’ filling in page numbers starting at 5. The index, length, match, and substr should not be confused with similar functions in the C Standard {7}; the awk versions deal with characters, while the C Standard {7} deals with bytes. To forestall any possible confusion, where strings are used as the name of a file or pipeline, the strings must be textually identical. The terminology ‘‘same string value’’ implies that ‘‘equivalent strings,’’ even those that differ only by <space>s, represent different files.

HISTORY OF DECISIONS MADE

This description is based on the new awk, ‘‘nawk,’’ (see The AWK Programming Language B21 ), which introduced a number of new features to the historical awk:

1. New keywords: delete, do, function, return
2. New built-in functions: atan2, cos, sin, rand, srand, gsub, sub, match, close, system
3. New predefined variables: FNR, ARGC, ARGV, RSTART, RLENGTH, SUBSEP
4. New expression operators: ?:, ˆ
5. The FS variable and the third argument to split are now treated as extended regular expressions.
6. The operator precedence has changed to more closely match C. Two examples of code that operate differently are: while ( n /= 10 > 1) . . . if (!"wk" ∼ /bwk/) . . .

Several features have been added based on newer implementations of awk:

1. Multiple instances of −f progfile are permitted.
2. New option: −v assignment
3. New predefined variable: ENVIRON
4. New built-in functions: toupper, tolower
5. More formatting capabilities added to printf to match the C Standard {7}.

Regular expressions have been extended somewhat from traditional implementations to make them a pure superset of Extended Regular Expressions as defined by this standard (see 2.8.4). The main extensions are internationalization

features and interval expressions. Traditional implementations of awk have long supported <backslash> escape sequences as an extension to regular expressions, and this extension has been retained despite inconsistency with other utilities. The number of escape sequences recognized in both regular expressions and strings has varied (generally increasing with time) among implementations. The set specified by the standard includes most sequences known to be supported by popular implementations and by the C Standard {7}. One sequence that is not supported is hexadecimal value escapes beginning with "\x". This would allow values expressed in more than 9 bits to be used within awk as in the C Standard {7}. However, because this syntax has a nondeterministic length, it does not permit the subsequent character to be a hexadecimal digit. This limitation can be worked around in the C language by the use of lexical string concatenation. In the awk language, concatenation could also be a solution for strings, but not for regular expressions (either lexical ERE tokens or strings used dynamically as regular expressions). Because of this limitation, the feature has not been added to POSIX. 2. When a string variable is used in a context where an ERE normally appears (where the lexical token ERE is used in the grammar) the string does not contain the literal slashes.

Some versions of awk allow the form: func name(args,. . . ) { statements } This has been deprecated by the language’s authors, who have asked that it not be included in the standard. Traditional implementations of awk produce an error if a next statement is executed in a BEGIN action, and cause awk to terminate if a next statement is executed in an END action. This behavior has not been documented, and it was not believed that it was necessary to standardize it. The specification of conversions between string and numeric values is much more detailed than in the documentation of traditional implementations or in The AWK Programming Language B21. Although most of the behavior is designed to be intuitive, the details are necessary to ensure compatible behavior from different implementations. This is especially important in relational expressions, since the types of the operands determine whether a string or numeric comparison is performed. From the perspective of an application writer, it is usually sufficient to expect intuitive behavior and to force conversions (by adding zero or concatenating a null string) when the type of an expression does not obviously match what is needed. The intent has been to specify existing practice in almost all cases. The one exception is that, in traditional implementations, variables and constants maintain both string and numeric values after their original value is converted by any use. This means that referencing a variable or constant can have unexpected side effects. For example, with traditional implementations the following program:

{ a = "+2" b = if (NR % 2) c = a + b if (a == b) print "numeric comparison" else print "string comparison" } would perform a numeric comparison (and output numeric comparison) for each odd-numbered line, but perform a string comparison (and output string comparison) for each even-numbered line. POSIX. 2 ensures that comparisons will be numeric if necessary. With traditional implementations, the following program: BEGIN { OFMT = "%e" print 3.14 OFMT = "%f" print 3.14 } would output 3.140000e+00 twice, because in the second print statement the constant 3.14 would have a string value from the previous conversion. The standard requires that the output of the second print statement be 3.140000. The behavior of traditional implementations was seen as too unintuitive and unpredictable. However, a further modification was made in Draft 11. It was pointed out that with the Draft 10 rules, the following script would print nothing: BEGIN { y[1.5] = OFMT = "%e" print y[1.5] } Therefore, a new variable, CONVFMT, was introduced. The OFMT variable is now restricted to affecting output conversions of numbers to strings and CONVFMT is used for internal conversions, such as comparisons or array indexing. The default value is the same as that for OFMT, so unless a program changes CONVFMT (which no historical program would do), it will receive the historical behavior associated with internal string conversions. The POSIX awk lexical and syntactic conventions are specified more formally than in other sources. Again the intent has been to specify existing practice. One convention that may not be obvious from the formal grammar as in other verbal descriptions is where <newline>s are acceptable. There are several obvious placements such as terminating a statement, and a backslash can be used to escape <newline>s between any lexical tokens. In addition, <newline>s without backslashes can follow a comma, an open brace, logical AND operator

(&&), logical OR operator (| |), the do keyword, the else keyword, and the closing parenthesis of an if, for, or while statement. For example: { print $1, $2 } The requirement that awk add a trailing <newline> to the program argument text is to simplify the grammar, making it match a text file in form. There is no way for an application or test suite to determine whether a literal <newline> is added or whether awk simply acts as if it did. Because the concatenation operation is represented by adjacent expressions rather than an explicit operator, it is often necessary to use parentheses to enforce the proper evaluation precedence. The overall awk syntax has always been based on the C language, with a few features from the shell command language and other sources. Because of this, it is not completely compatible with any other language, which has caused confusion for some users. It is not the intent of this standard to address such issues. The standard has made a few relatively minor changes toward making the language more compatible with the C language as specified by the C Standard {7}; most of these changes are based on similar changes in recent implementations, as described above. There remain several C language conventions that are not in awk. One of the notable ones is the comma operator, which is commonly used to specify multiple expressions in the C language for statement. Also, there are various places where awk is more restrictive than the C language regarding the type of expression that can be used in a given context. These limitations are due to the different features that the awk language does provide. This standard requires several changes from traditional implementations in order to support internationalization. Probably the most subtle of these is the use of the decimal-point character, defined by the LC_NUMERIC category of the locale, in representations of floating point numbers. This locale-specific character is used in recognizing numeric input, in converting between strings and numeric values, and in formatting output. However, regardless of locale, the period character (the decimal-point character of the POSIX Locale) is the decimal-point character recognized in processing awk programs (including assignments in command-line arguments). This is essentially the same convention as the one used in the C Standard {7}. The difference is that the C language includes the setlocale() function, which permits an application to modify its locale. Because of this capability, a C application begins executing with its locale set to the C locale, and only executes in the environment-specified locale after an explicit call to setlocale(). However, adding such an elaborate new feature to the awk language was seen as inappropriate for POSIX. 2. It is possible to explicitly execute an awk program in any desired locale by setting the environment in the shell. The behavior in the case of invalid awk programs (including lexical, syntactic, and semantic errors) is undefined because it was considered overly limiting on implementations to specify. In most cases such errors can be expected to produce a diagnostic and a nonzero exit status. However, some implementations may choose to extend the language in ways that make use of certain invalid constructs.

Other invalid constructs might be deemed worthy of a warning but otherwise cause some reasonable behavior. Still other constructs may be very difficult to detect in some implementations. Also, different implementations might detect a given error during an initial parsing of the program (before reading any input files) while others might detect it when executing the program after reading some input. Implementors should be aware that diagnosing errors as early as possible and producing useful diagnostics can ease debugging of applications, and thus make an implementation more usable. The unspecified behavior from using multicharacter RS values is to allow possible future extensions based on regular expressions used for record separators. Historical implementations take the first character of the string and ignore the others. The undefined behavior resulting from NULs in regular expressions allows future extensions for the GNU gawk program to process binary data. Unspecified behavior when split(string,array,<null>) is used is to allow a proposed future extension that would split up a string into an array of individual characters.