Awk

lastmod 2026-02-06created 2014-10-1729 edits

A programming language that runs without compilation. The name comes from the initials of its three co-developers: A. V. Aho, P. J. Weinberger, and B. W. Kernighan. Based on the observation that complex languages such as C are in practice used mostly for simple tasks like organizing data or documents, it was simplified to make data processing more convenient. It scans the input file, reads it in units of a predefined record separator, and gives each field a regular name using the field separator. By default, the record separator is a return (new line) and the field separators are blank and tab (tap); the record, i.e. all fields, is named $0, the first field $1, the second $2, and so on. It works with a simple structure in which each record is read and a programmed action is performed according to the pattern in its fields, and it is used for data validation, conversion, output in various formats, building databases, and more.

Options

-F: Specifies the field separator.; ex) awk -F"." ' /txt/ { print "mv " $1 "." $2 " " $1 ".doc" } '

-v 변수=값: Assigns a value to a variable before the script runs.

-f: Specifies a script file.

Structure

Start (BEGIN): Stage that converts the input data into a form suitable for execution
Execution (Routine): Stage that processes the well-organized (normalized) data in the actual routine
End (END): Additional output of results after the data has been processed

Built-in variables

System variable	Description
FILENAME	Current file name
FS	Input field separator (default: `' '` - space -)
NF	Number of fields in the current record
NR	Current record number
OFMT	Output format for numbers (default: `$.6g`)
OFS	Output field separator (default: blank line)
ORS	Output record separator (default: newline)
RS	Input record separator (default: newline)
$0	Input record
$n	Nth field of the input record
ARGC	Number of command-line arguments
ARGV	Array containing the command-line arguments
ENVIRON	Associative array holding the environment variables
FNR	Same as NR, except that it applies to the current file
RSTART	Start of the string that satisfies the given match operation
RLENGTH	Length of the string that satisfies the given match operation

Operators

Arithmetic operators	`=`, `+=`, `-=`, `*=`, `/=`, `%=`
Conditional operator	`(condition)? (true) : (fales)`
Logical operators	`\|\|`, `&&`, `!`
Pattern operators	`~`, `!~`
Comparison operators	`<`, `<=`, `>`, `>=`, `!=`, `==`
Increment/decrement operators	`++`, `--`
Field reference	`$`

Control statements

break
continue
do { Routine } while (condition)
exit
for ( init ; condition ; re ) { Routine }
if ( condition ) { Routine } else { Routine }
while (condition) { Routine }
return

Functions

Neutron star collision energy :: awk

String functions

Function name	Description
`gsub(r, s)`	Replaces regular expression r with string s throughout the entire input string.
`gsub(r, s1, s2)`	Replaces regular expression r with string s1 in input string s2.
`index(s1, s2)`	Returns the position of s2 in s1. 0 if not found
`length(arg)`	Returns the length of the argument.
`match(s, r)`	Returns the position of the part of string s that matches regular expression r.
`split(string, array[, seperator])`	Splits the given string into an array by the separator (default: space).
`sub(r, s) , sub(r, s1, s2)`	Same as gsub, but even if several strings match the regular expression, only the first one is replaced
`substr(s, m)`	Returns the string from position m to the end of string s.
`substr(s, m, n)`	Returns the string from the m-th to the n-th position of string s
`tolower(string)`	Converts uppercase letters to lowercase.
`toupper(string)`	Converts lowercase letters to uppercase.

Math functions

Function name	Description
`atan2(x, y)`	archtangent value
`cos(x)`	cos value
`exp(x)`	Power of e, the base of the natural logarithm
`int(x)`	Returns as an integer
`log(x)`	Logarithm
`rand()`	Random number from 0 to 1
`sin(x)`	sin value
`sqrt(x)`	Square root
`srand(expr)`	Random number with the argument (if omitted, a random number is generated from the time)

I/O functions

Function name	Description
`close(filename)`	Closes the given file.
`close(cmd)`	Closes the given command pipe.
`delete array[element]`	Deletes the given array element.
`getline()`	Reads the next record.
`getline [variable] [< "filename"]`	Reads from a file.
`next`	Takes the next record as input.
`print [args] [> "filename"]`	Prints the arguments
`printf "format" [,exp] [> "filename"]`	Prints the arguments according to the format.
`sprintf(format [,exp])`	Used the same way as printf, but only returns the value without printing it.
`system(cmd)`	Runs an internal system command.

How to join multiple lines of output

You can use the following tools:

TODO
Needs to be tested

awk ' /LABEL/ {f=3} f-->0 {$1=$1;printf "%s ",$0} f==0 {print ""}' file

Example

GCC Include directory

A bash sample for viewing the list of GCC's INCLUDE directories is as follows.

$ echo | gcc -Wp,-v -x c - -fsyntax-only 2>&1 \
    | awk 'BEGIN{FS="[ \t]";RS="[\r\n]";s="#include <...> search starts here:";e="End of search list.";i=0;}{if($0!=""){if($0==e){i=0;}if(i==1){print($0);}if($0==s){i=1;}}}'

For reference, the cleaned-up AWK syntax is as follows.

BEGIN {
    FS = "[ \t]";
    RS = "[\r\n]";
    s = "#include <...> search starts here:";
    e = "End of search list.";
    i = 0;
}
{
    if ($0 != "") {
        if ($0 == e) {
            i = 0;
        }
        if (i == 1) {
            print($0);
        }
        if ($0 == s) {
            i = 1;
        }
    }
}
END {
}

Column print

$ cat text.txt | awk '{printf("%-50s %-50s\n", $2, $3);}'

Text Processing Commands

List of text processing commands

Text	Common	grep, sed, awk, cat, xargs, rev, tee, ack, tr, stdbuf, split
Diff	diff, patch
Cutting	cut, head, tail, less, more, split, watch
Sort	uniq, sort
Hash	base64, md5sum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum
Clipboard	clip (Windows), pbcopy (Mac OSX), pbpaste (Mac OSX), xclip (Debian)
E.T.C	regexp, parallel

Favorite site

Wikipedia (en) description of awk
The GNU Awk User's Guide ¹
AWK usage
Wikibooks: GNU Awk User's Guide/Functions
IBM technical library: Introduction to the AWK language ²
IBM technical library: UNIX tips and tricks for beginners, Part 3: Filters and regular expressions: Using awk on the command line ³
KLDP description of AWK ⁴
Detailed usage of AWK ⁵
AWK & SED chunk_1 ⁶
Batch-renaming files (in a format such as ([a-z]+)-([0-9]+).([a-z]+))
AWK shell programming ⁷

Tutorials

[Recommended] AWK script guide - mug896 ⁸ (archive)

References

Gnu-software-gawk.zip ↩
Www.ibm.com-au-gawk.zip ↩
Www.ibm.com-au-unixtips3.zip ↩
An_awk_primer.pdf ↩
Comefeel_awk.pdf ↩
Awk_sed_chunk_1.pdf ↩
Awk_shell_programming.pdf ↩
Mug896-awk-script-master-fd0223a.zip (2020-10-07) ↩

author Zer0 sitemap

Awk