Notes to self

Benchmarking Ruby 3 in PDF generation

Ruby 3 JIT was optimized against CPU-intensive demo, which left Rails enthusiasts left out. But what about tasks in the middle of both? Here’s my latest benchmark on Ruby 3 JIT in generating PDF files with Prawn.

A little over a year ago, I created a PDF generation benchmark to help Takashi optimize the upcoming Ruby 3 JIT. I blogged about how to make such a benchmark, and my results for Ruby 2.5, 2.6, and 2.6 –jit.

The JIT wasn’t very successful for this task, but it was very early. So, what about now, that Ruby 3 is out?

My numbers are again for generating a PDF invoice with InvoicePrinter, this time for version 2.1 that supports Ruby 3.

Here are CPU numbers:


/home/strzibny/.rubies/ruby-2.7.0/bin/ruby:        64.6 i/s
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby:        61.8 i/s - 1.05x  slower
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby --jit:  56.3 i/s - 1.15x  slower

/home/strzibny/.rubies/ruby-2.7.0/bin/ruby:        64.9 i/s
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby:        62.0 i/s - 1.05x  slower
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby --jit:  54.5 i/s - 1.19x  slower

As you can see, we are not there yet. Ruby 3.0 is up to 5% slower than 2.7, and Ruby 3 JIT up to 19% slower.

And numbers for memory:


/home/strzibny/.rubies/ruby-3.0.0/bin/ruby:  23896000.0 bytes
/home/strzibny/.rubies/ruby-2.7.0/bin/ruby:  25920000.0 bytes - 1.08x  larger
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby --jit:  79876000.0 bytes - 3.34x  larger

/home/strzibny/.rubies/ruby-3.0.0/bin/ruby:  24140000.0 bytes
/home/strzibny/.rubies/ruby-2.7.0/bin/ruby:  26108000.0 bytes - 1.08x  larger
/home/strzibny/.rubies/ruby-3.0.0/bin/ruby --jit:  79916000.0 bytes - 3.31x  larger

It seems we are at least getting better RAM consumption in Ruby 3.0, Ruby 2.7 takes up to 8% more memory.

As expected, Ruby JIT takes a lot more memory – more than three times. I wonder if somebody runs the Ruby 3.0 JIT in some real workloads because I cannot see myself using the JIT any time soon.

by Josef Strzibny