Time individual statements in bash

Sometimes the 1000+ line bash script you inherited at work takes 3 hours to complete and you’re not sure where the time is being spent. StackOverflow has some good answers on this topic:

http://stackoverflow.com/questions/5014823/how-to-profile-a-bash-shell-script
http://stackoverflow.com/questions/18039751/how-to-debug-a-bash-script-and-get-execution-time-per-command

The simplest one that gives me the most value for the least effort (imo) is:

PS4='+[${SECONDS}s][${BASH_SOURCE}:${LINENO}]: ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
#!/bin/bash

PS4='+[${SECONDS}s][${BASH_SOURCE}:${LINENO}]: ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
set -x

echo 1
sleep 1
echo 2
sleep 2
echo 3

This produces the following output:

joao@home:~> ./timings.sh
+[0s][./timings.sh:7]: echo 1
1
+[0s][./timings.sh:8]: sleep 1
+[1s][./timings.sh:9]: echo 2
2
+[1s][./timings.sh:10]: sleep 2
+[3s][./timings.sh:11]: echo 3
3

Happy bash profiling !

Time individual statements in bash

Redirect STDOUT/STDERR

Sometimes you want to redirect your shell script’s STDOUT/STDERR to a log file. This is pretty simple to do, ie:

./myscript.sh &> output.log

But sometimes you want to do the redirect inside the script itself ( ie, maybe redirecting is conditional, or maybe the file name is conditional on something only the script knows about)


#!/bin/bash

exec >> output.log
exec 2>&1

>&2 echo "This was printed to stderr"
echo "This was printed to stdout"

Or if you want to write to both a file and the terminal:

#!/bin/bash

exec 1> >(tee -a output.log)
exec 2>&1

>&2 echo "This was printed to stderr"
echo "This was printed to stdout"

Stackoverflow has other ways to do it:

Redirect STDOUT/STDERR

Wait for a file in bash

Sometimes your bash script needs to wait for a file to exist before proceeding. Here is one way to do it.

function wait_till_exists {
    file=$1
    timeout=${2:-60}
    counter=1
    until ls -l $file &> /dev/null
    do
        if [ $counter -gt $timeout ]; then
            return 1
        fi
        sleep 1
        ((counter++))
    done
    return 0
}

wait_for_file="/tmp/myfile"
if ! wait_till_exists "$wait_for_file" 5; then
    echo Timed out waiting for $wait_for_file to exist
exit 1
fi

echo "$wait_for_file found"
Wait for a file in bash