Bash Command awk Cheat Sheet

A cheat sheet for the awk command that runs on the Bash command-line language in Linux. Introduces everything from simple references to advanced topics like external command execution.

Note that the concepts of records and columns are the same as in Excel tables.

• Record … row. From top to bottom
• Column … column. From left to right

Note that awk has the following variables prepared:

• FS separator (default is space)
• NF total number of columns
• NR total number of records
• $1 first column
• $2 second column

Output lines matching regex with grep -E

The following is an example of outputting lines matching a regex with grep -E.

awk '/regex/'

Output lines where the first column matches regex

The following example outputs lines where the first column matches the regex.

awk '{if ($1 ~ /regex/){print $1,$2,$3,$4}}'

The following outputs lines where the first column matches the regex from the output of the df command.

df | awk '{if ($1 ~ /\w{1,2}:/){print $1,$2,$3}}'

Execute python / Execute external commands

The following is an example of multiplying the value in the fourth column by 100 using python.

awk '{"python -c \"print("$4"*100)\" | perl -pe 'chomp'" |& getline $11 ; print "The 4th column was being x100:"$11 }}'

The following example multiplies the value in the fourth column by 100 using python.

$ df 2>/dev/null | tail -3
E:             2930133932  381876300 2548257632  14% /mnt/e
G:             2930133932 1545875952 1384257980  53% /mnt/g
H:             2930134012 2625853000  304281012  90% /mnt/h

$ df 2>/dev/null | tail -3 | awk '{if ($1 ~ /\w{1}:/){"python -c \"print("$4"/1000/1000)\" | perl -pe 'chomp'" |& getline $11 ; print $1,$11"(GB)",$5,$6}}'
E: 2548(GB) 14% /mnt/e
G: 1384(GB) 53% /mnt/g
H: 304(GB) 90% /mnt/h

Important points here:

• When passing $1 etc. as arguments to commands, don’t enclose them in "".
• Most outputs include \n newline, so chomp with perl etc.
• With |& getline $11, you can assign command output to $11. Without this, it’s processed as $0 (?), and the output is inserted at the beginning of columns as the 0th column.
• When doing assignment processing, system() can’t (?) be used

awk - Assigning system command’s output to variable - Stack Overflow

Kill all matching processes

The following command kills processes containing npm and outputs the execution result.

ps aux | grep "npm" | awk '{"kill -9 "$2 |& getline $11 ; print $11}'

The following is functionalized:

# killp(){
#     ps aux | grep "$1" | awk '{"kill -9 "\\$2 |& getline \\$11 ; print \\$11}'
# }
killp(){
    ps aux | grep "$1" | awk '{"kill -9 "$2 |& getline $11 ; print $11}'
}

Kill all matching background processes

The following command kills those containing webpack-dev.

ps S | awk '/webpack-dev/' | awk '{"kill -9 "$1 |& getline $11 ; print $11}'

Please also refer to these:

• AWK Cheat Sheet | ShortcutFoo
• AWK cheatsheet