Linux

Bash tips

Evan Davies

Aug 21, 2008 • 11 min read

Originally from here.

Introduction

If you type in a rather long and complex command at the command line, you might want to save that command in a text file, so you can execute it later without having to type the whole long thing in again. If you add a line to the beginning of that file, like “#!/bin/bash” (a so-called “shebang” line), and then make the file executable with the chmod command, you’ve just created a shell script.

Here’s an example shell script:

#!/bin/bash
# Any line that starts with a hash, besides the shebang line, is a comment.
echo "Hello world!"

You run the shell script just like any other executable. Some folks create a bin directory in their home directory within which to place their scripts. Others drop them in /usr/local/bin.

Some key points to keep in mind about shell scripts:

They execute their commands, line by line, just as if you’d typed them in yourself at the command line.
The commands in the script run in a subshell of the shell from which you executed the script.
If you want the script to affect your present working directory or environment variables, source it with . scriptname.
Don’t put spaces around equal signs.

You can check out the bash page and you’ll also find lots of info in the Advanced Bash-Scripting Guide.

Technologies

Using quotes

Quotation marks, either double quotes (“) or single quotes (‘), are often needed in shell scripts to pass arguments that have spaces or other special characters. The two quotes have slightly different meanings. Single quotes protect the exact text they enclose.

Inside double quotes, the dollar sign ($) is still special, so shell variables and other $ expressions are interpreted, as is the backquote (`) used for command substitution. Lastly, the backslash (\) is special in double quotes, so be cautious. Some examples and the shell output of each:

$ echo 'The location of the shell is stored in $BASH'
The location of the shell is stored in $BASH
$ echo "The location of the shell is $BASH"
The location of the shell is /bin/bash
$ echo '\\'
\\
$ echo "\\"
\

Return values

The return value, also known as error code, delivers the exiting state of the most recently executed program. It is defined to be 0 in case no error happened. It is stored in the variable $?:

scorpio:~ # echo "hello world"
hello world
scorpio:~ # echo $?
0
scorpio:~ # ls /doesNotExist
/bin/ls: /doesNotExist: No such file or directory
scorpio:~ # echo $?
2

The [ ] operators deliver 0 if they contain a true expression, 1 if they contain a false expression:

scorpio:~ # [ 5 = 5 ]
scorpio:~ # echo $?
0
scorpio:~ # [ 5 = 6 ]
scorpio:~ # echo $?
1
scorpio:~ #

Here, [ is a command. A command requires a blank behind it – thus the following error is very common:

scorpio:~ # [5 = 6]
bash: [5: command not found

Also the ] requires a blank in front of it:

scorpio:~ # [ 5 = 6]
bash: [: missing `]'

The “=” sign requires a blank in front of it and after it. Otherwise, the bash only sees “something” within the brackets, and that is true (0), while “nothing” is false (1):

scorpio:~ # [ aoei ]
scorpio:~ # echo $?
0
scorpio:~ # [ 5=5 ]
scorpio:~ # echo $?
0
scorpio:~ # [ ]
scorpio:~ # echo $?
1
scorpio:~ # true
scorpio:~ # echo $?
0
scorpio:~ # false
scorpio:~ # echo $?
1

You can hand over your own error codes with the command exit. This command will exit from the current process. As the ( ) signs spawn out an own process, you can do the following:

scorpio:~ # (exit 5); echo $?
5

Conditions

Conditions are handled with the if command, using the form if <expression> then command1 else command2 fi “fi” as the reverse of “if”, closes the loop. bash allows you to substitute newlines with “;”, so the following form is also legal: if <expression>; then command1; else command2; fi Example:

scorpio:~ # if true; then echo "yes"; else echo "no"; fi
yes
scorpio:~ # if false; then echo "yes"; else echo "no"; fi
no

You can omit the else-branch:

scorpio:~ # if [ 5 = 5 ]; then echo "five is five"; fi
five is five
scorpio:~ # export name=Thorsten
scorpio:~ # if [ $name = Thorsten ]; then echo "it's you"; fi
it's you

In this case, bash replaces $name with its content. That means, we get an error if $name is empty:

scorpio:~ # export name=
scorpio:~ # if [ $name = Thorsten ]; then echo "it's you"; fi
bash: [: =: unary operator expected
scorpio:~ # if [  = Thorsten ]; then echo "it's you"; fi
bash: [: =: unary operator expected

To overcome this problem, one can add an “x” to the left and to the right:

scorpio:~ # export name=Thorsten
scorpio:~ # if [ x$name = xThorsten ]; then echo "it's you"; fi
it's you
scorpio:~ # export name=
scorpio:~ # if [ x$name = xThorsten ]; then echo "it's you"; fi
scorpio:~ #

Negations

You can invert true/false values with the ! operator:

$ ! true
$ echo $?
1

That helps you if you want to do “not”-statements:

if ! grep "network error" /var/log/messages
then echo "There is no network error mentioned in the syslog"
fi

The ! is just like a command with arguments, that’s why it is important to have a space behind it:

$ !true
bash: !true: event not found

Loops

You can loop on a condition:

while [ 5 = 5 ]; do echo "This never stops"; done

until [ 5 = 6 ]; do echo "This never stops"; done

$-variables

$! : PID of last process that has been started in the background (e.g. firefox &)

$? : return value of last commAnd

Get your IP address

A bash example to get your IP address using http://www.whatismyip.com:

lynx -dump http://www.whatismyip.com | grep -i "Your IP is" | awk '{ print $4; }'

Or use this:

$ ifconfig ppp0 | awk '/inet addr/{print $2}' | cut -d: -f2

Or this:

$ ifconfig ppp0 | sed -rn 's/^.*inet addr:([^ ]+).*$/\1/p'

Note that ppp0 stands for the first point-to-point-protocol device which is usually a modem. If you are using an ethernet adapter instead, replace ppp0 with eth0. You can also leave it out all together to list all the addresses associated with your computer.

Some counting examples

x=0; ((x++))
x=0; x=$((x+1))
x=0; let x=$x+1
for (( x=0; x<10; x++ )) ; do echo $x ; done

for x in $(seq 0 9); do echo $x; done
x=0; x=$(expr $x + 1)
x=0; x=$(echo $x + 1|bc)
x=4; x=$(echo scale=5\; $x / 3.14|bc)

The first four demonstrate Bash’ internal counting mechanisms, these will not use external programs and are thus safe (and fast). The last four use external programs. bc, as used in the last two examples, is the only one that supports numbers with decimals. For adding, subtracting and multiplying, you don’t have to do anything special, but for division you need to specify the number of digits to keep (default is none) using the ‘scale=n’ line.

Combining multiple conditions of if statements

AND:

if((a==1 && b==2))

OR:

if(( a!=1 || b>10 ))

Conditional execution

As stated above, you can do something like

scorpio:~ # if (( 5 == 5 && 6 == 6 )); then echo "five is five and six is six"; fi
five is five and six is six

We call 5 == 5 the left term and 6 == 6 the right term. Bash first evaluates the left term and if it is true, it evaluates the right term. If both are true, the result is true, otherwise, the result is false (boolean logic). It is an interesting effect that, if the left term is false, the right term is not even evaluated – the result will be false in any case. That means, if we replace the terms by commands, the right command will only be executed if the left command succeeded. One can use this effect:

scorpio:~ # ls > /tmp/dir && echo "the command succeeded"
the command succeeded
scorpio:~ # ls > /proc/cmdline && echo "the command succeeded"
/bin/ls: write error: Input/output error
scorpio:~ #

In the above example, bash tells you if the command succeded. The right command is conditionally executed. Considering a famous saying, you can even program a bird using this technology:

eat || die

This line says “eat or die” which is very useful if you want a program to exit in case it cannot perform a certain operation:

touch /root/test || exit

Functions

You can define functions for structured programming. It is also possible to hand over parameters to a function. Example:

#!/bin/bash
# This program calculates the square of a given number 

function square \
{
  echo "The square of your number is "
  echo $((a*a))
} 

echo "Input a number"
read a
square $a

UseCases

Renaming a set of files

The following example will rename all files ending in .jpeg to end in .jpg

$ for file in *.jpeg; do mv $file ${file%.jpeg}.jpg; done

mmv is a really nifty tool for doing this sort of thing more flexibly too. The rename command also provides similar functionality, depending on your linux distribution.

If you have a load of files in all capital letters (common if you unzip old DOS programs for instance), you can use

 $ for file in *; do mv $file $(echo $file | tr [[:upper:]] [[:lower:]]); done

to make all the files in the current directory lowercase.

Converting several wav files to mp3

This statement will convert all .wav files in a directory to mp3 (assuming you already have the lame package installed):

for i in *.wav; do lame -h $i && rm $i; done

The double ampersand prevents rm from deleting files that weren’t successfully converted.

Full file name

Prints the full filename removing ./ and ../ and adding `pwd` if necessary.

fqn.sh

#!/bin/sh

# return the full filename, removing ./ ../ adding `pwd` if necessary

FILE="$1"

# file		dot relative
# ./file	dot relative
# ../file	parent relative
# /file		absolute
while true; do
	case "$FILE" in
		( /* )
		# Remove /./ inside filename:
		while echo "$FILE" |fgrep "/./" >/dev/null 2>&1; do
			FILE=`echo "$FILE" | sed "s/\\/\\.\\//\\//"`
		done
		# Remove /../ inside filename:
		while echo "$FILE" |grep "/[^/][^/]*/\\.\\./" >/dev/null 2>&1; do
			FILE=`echo "$FILE" | sed "s/\\/[^/][^/]*\\/\\.\\.\\//\\//"`
		done
		echo "$FILE"
		exit 0
		;;

		(*)
		FILE=`pwd`/"$FILE"
		;;
	esac

done

Resolving a link

This little script follows symbolic links wherever they may lead and prints the ultimate filename (if it exists!) (note – it uses the previous script fqn.sh)

#!/bin/sh

# prints the real file pointed to by a link
# usage: $0 link

ORIG_LINK="$1"
LINK=`fqn "$1"`
while [ -h "$LINK" ]; do
	DIR=`dirname "$LINK"`

	# next link is everything from "-> "
	LINK=`ls -l "$LINK" |sed "s/^.*-> //"`

	LINK=`cd "$DIR"; fqn "$LINK"`
	if [ ! -e "$LINK" ]; then
		echo "\"$ORIG_LINK\" is a broken link: \"$LINK\" does not exist" >&2
		exit 1
	fi
done
echo "$LINK"
exit 0

This is a lot simpler, and just as useful (but doesn’t do dead link checking)

#!/bin/bash
ORIG_LINK="$1"
TMP="$(readlink "$ORIG_LINK")"
while test -n "$TMP"; do
    ORIG_LINK="$TMP"
    TMP="$(readlink "$ORIG_LINK")"
done
echo "$ORIG_LINK"

Viewing a file according to its type

I save this as a script called v and it is my universal viewing script – it can be extended to any kind of file and to use your favourite tools for each type of file. It uses the file utility to determine what sort of file it is and then invokes the correct tool. It automatically adapts to being used on the system console or under X.

(note – this uses a prior script fqn.sh)

#! /bin/sh
# View files based on the type of the first one:
# Relies partly on 'less' to invoke an appropriate viewer.
# Make sure than you have this set:
# LESSOPEN="|lesspipe.sh %s"

FIRST="$1"
FULL=`fqn $FIRST`

# if we are being piped to, just run less
[ -z "$FIRST" -o ! -r "$FIRST" ] && exec less "$@"

# some file types beyond what /usr/bin/lesspipe.sh handles:
case "$FIRST" in
  *.cpio.bz2) bunzip2 -c $1 | cpio -ctv 2>/dev/null |less; exit ;;
  *.cpio) cpio -ctv $1 2>/dev/null |less; exit;;
esac

TYPE=`file -L -i "$FIRST"  2>/dev/null | \
  awk '{len=length("'"$FIRST"'")+ 2; $0=substr($0, len); print $1;}'`
echo "Type=\"$TYPE\""
case "$TYPE" in
    image/jpeg | image/tiff | image/gif | image/x-xpm | image/*)
        xzgv "$@" ;;
    application/pdf)
	XPDFGEOM=`xdpyinfo |awk '/dimensions/ {print $2}'| \
          awk -Fx '{printf("%dx%d+0+0", 0.60*$1, $2-35)}'`
	xpdf -geometry $XPDFGEOM -z width "$@"
	;;
    application/postscript)
        gv "$@" ;;
    application/msword)
        if [ -z "$DISPLAY" ] ; then antiword "$@" | \
          less; else soffice "$@" & fi ;;
    text/html)
        if [ -z "$DISPLAY" ] ; then lynx "$@"; else dillo "$FULL" & fi ;;
    audio/mpeg)
        mpg123 "$FIRST" ;;
    *)
        less "$@" ;;
esac

Finding the 50 largest directories

du -S / | sort -nr | head -n50

Auto-generating a shebang line

(echo -n '#!'; which perl; tail -n+2 foo) > bar

This will print “#!” and the path to perl, then everything except the first line (the original shebang) of the file foo; and redirect all that to go into a file bar.

This is really meant for use in install scripts; obviously you know where perl is on your system, but not necessarily on anybody else’s system (though /usr/bin/perl is a safe guess). But you know bash is always at /bin/bash. So, instead of just saying in the documentation “change the first line …..”, you can write a little shell script that puts the correct shebang line in the file and copies it to /usr/local/bin. Then, put this script in a .tar.gz file with your perl script and any documentation.

Creating an audio CD from .mp3 files

This little script requires the mpg123 and cdrdao packages. It uses mpg123 to create a .wav file from each mp3 in the current directory, and builds up a “TOC” file (Table Of Contents) as it goes along, using the && notation to be sure only to include successfully-converted files. Finally, it starts writing the CD.

The TOC file is given a name based on the PID, and is placed in the current directory. Neither it nor the .wav files will be deleted at the end of the script, so you can always burn another copy of the CD if you want.

#!/bin/sh
tocfile="cd_$$.toc"
echo "CD_DA" > $tocfile
for i in *mp3
 do wav="`basename $i .mp3`.wav"
    mpg123 -w$wav $i\
    && echo -en >>$tocfile "TRACK AUDIO\nFILE \"$wav\" 0\n"
done
echo -e "TOC file still available at $tocfile"
cdrdao write $tocfile

Note: in the for statement, you might think at first sight that it should be for i in *.mp3; but Linux doesn’t care about filename extensions at all. * will match a . character; therefore, *mp3 will match everything ending in mp3 or .mp3. Those characters are unlikely to be found on the end of anything but an mp3 file.

Portable Indirection

Sometimes you need to use the value of a variable whose name is held in another variable. This is indirection. For example:

 for V in PATH LD_LIBRARY_PATH; do echo ${!V}; done

but that isn’t portable to older shells, eg. the old Bourne shell. A portable way is to use the following instead of ${!V}:

 `eval echo \\$"$V"`

It’s not pretty but at least it works on nearly all Unixes.

[ someone said – “It does require that the variable be exported.” Actually no. It works without exporting V on bash and on Solaris sh ]

Matching Curly Brackets

If you indent your C, Perl or PHP code in what is commonly called “K&R style”, which is to say something like this

# nested loop example
for ($i = 0; $i <= 6; ++$i) {
    for ($j = 0; $j <= $i; ++$j) {
        print "$i : $j ";
    };
    print "\n";
};

i.e. where the opening curly bracket appears on the same line as the flow-control keyword (if, while &c.) and everything up to, but not including, the closing curly bracket is indented, then you may find this useful. I came up with it out of necessity, and thought it was just too important not to share. All it does is display the lines with a { but no following } (in Perl, these denote associative array indexes) or a } with no preceding {. In other words, just the opening and closing lines of loops, not the content. This can help you to spot those annoying missing-bracket errors.

awk '/({[^}]*$)|(^[^{]*})/{print}' foo

Replace foo with the filename to be examined. Or, of course, you can omit the filename and use the command in a pipeline.

The above example would appear as

for ($i = 0; $i <= 6; ++$i) {
    for ($j = 0; $j <= $i; ++$j) {
    };
};

rpminfo

This is a small script that prints useful info about an rpm package. It was inspired by the ability of less to give almost useful info about an rpm file through lesspipe.sh.

#!/bin/sh
echo INFORMATION: ; rpm -qi $* ; echo
echo REQUIRES: ; rpm -qR $* ; echo
echo PROVIDES: ; rpm -q --provides $* ; echo
echo FILELIST: ; rpm -qlv $*

Save it to a file called rpminfo in your path and chmod it so it’s executable. It is often useful to pipe the output to less, as in the examples below.

To get info on an installed package, you can just give the package name like:

rpminfo XFree86 |less

For a package file that you have just downloaded, give the -p command option, just like rpm:

rpminfo -p XFree86-4.1.0-25.i386.rpm |less