Notes to self

Benchmarking programs with /usr/bin/time

If you ever used the time to measure the program’s execution, you might want to know how to improve your results by running with a higher process priority.

time vs /usr/bin/time

First of all, it’s important to realize that running time and /usr/bin/time might not be the same thing. While time should be running the first program on $PATH, it doesn’t necessarily end up running /usr/bin/time:

$ time

real  0m0.000s
user  0m0.000s
sys 0m0.000s

$ /usr/bin/time
/usr/bin/time: missing program to run
Try '/usr/bin/time --help' for more information.

That is because time is also a Bash function (on Fedora-based systems), which can also benchmark your program run, but lacks the detailed -v (as “verbose”) option, among other things.

So ideally, you want to make sure to benchmark programs as:

$ /usr/bin/time -v $PROGRAM

More accurate benchmarks

Because Linux tries to give all running processes access to CPU, there will be a lot of context switching involved during your benchmark. chrt is a small program to manipulate the real-time attributes of a given process that can help us combat it. With its -f option, we can change the FIFO priority to give the process under benchmark the priority it needs. We prepend the whole time command with chrt -f 99:

$ sudo chrt -f 99 /usr/bin/time -v $PROGRAM

Now we get more accurate results when benchmarking with time.

Alternatives

The time program is not the only program we can use. A good alternative to /usr/bin/time is perf stat for which we need the perf package (on Fedora):

$ sudo dnf install -y perf

Again, we should run it with chrt -f 99 and -ddd to get the most accurate and detailed report:

$ sudo chrt -f 99 perf stat -ddd $PROGRAM

perf stat can also show cache misses or instructions-per-cycle.

Work with me

I have some availability for contract work. I can be your fractional CTO, a Ruby on Rails engineer, or consultant. Write me at strzibny@strzibny.name.

RSS