Jeremy Evans (of Roda and Sequel fame) created an interesting benchmark comparing memory and runtime performance of large number of routes (up to 10000) in various Ruby web frameworks. I was immediately interested and ran it to get the numbers.
Jeremy’s benchmark is called r10k and supports several web frameworks. Since the results are not included in the original repository I ran this benchmark on my laptop (5th gen i7 ThinkPad Carbon) to find out.
I used latest stable versions of Roda (3.16.0) and Sinatra (2.0.5) together with Rails 6 (6.0.0.beta1) all running Ruby 2.6.0. I did only small tweaks to the original benchmark by upgrading Rails and removing config.secret_token option. Here are results for initial memory taken:
And here once again as numbers:
app 10 100 1000 10000 <- Number of routes
roda 15884 15560 17504 28760
sinatra 23332 24304 29740 100152
rails 41592 43148 52172 115076
If you are familiar with Roda you are not surprised that it performs the best, but what’s interesting is that by using large number of routes the memory gets worse for Sinatra way faster than for Rails. With 10000 routes Rails takes only 15 MB more memory than Sinatra and with even more routes Rails would win here in terms of memory used. The runtime performance is even more interesting:
And once again pure numbers:
app 10 100 1000 10000
roda 0.172025133855641 0.256997955031693 0.338058794150129 0.446734027005732
sinatra 1.71625644806772 2.67249810113572 11.123352029128 152.814259551931
rails 8.51651063212194 8.59500932297669 9.16402177000418 10.777155773947
Roda performance is again not surprising to me. It’s rock solid and fast. Rails is also pretty good. Yes, it’s slow, but it doesn’t get much worse with many routes and the performance is predictable. Unfortunately Sinatra performs poorly with 10000 routes and more. There is a real jump there and only gets worse with bigger numbers.
If we take a look on the application code generated by the benchmark we can see that it’s not just about number of routes, but nesting level. Each level of magnitude introduce more complex routes by extra directory level. This is snipped for those 10000 routes:
require 'sinatra/base'
class App < Sinatra::Base
get '/a/a/a/a' do
'4797479747974797'
end
get '/a/a/a/b' do
'4797479747974798'
end
get '/a/a/a/c' do
'4797479747974799'
end
get '/a/a/a/d' do
'47974797479747100'
end
...
end
So this does not automatically mean that Sinatra doesn’t handle big number of routes well, but that there are performance issues given large number of more nested routes. I am not familiar with Sinatra routing internals so please share if you know more (or if this benchmark has a bug).
Finally this unfortunately still does not say much about routes in your application. As you can see these routes are not dynamic and not working with any parameters so take everything with a pinch of salt.
Get Test Driving Rails and make your tests faster and easier to maintain.
