Benchmarking Rails 6 vs. Rails 7

This blog post is a continued work of Benchmarking Ruby 3.1 (+YJIT) vs. Ruby 2.7. We wanted to check the performance of Rails 6.1 and Rails 7.0 compared to Rails 6.0, which we are currently on.

With the help of hey, we used a similar approach to benchmarking Puma as the last time.

We ran all benchmarks on DigitalOcean CPU-Optimized, 2 vCPUs, 4 GB, Ruby 2.7.2, and Rails, all booted in production mode. As Rails 7 prefers Ruby 3+, we tested Rails 7.0 on Ruby 3.1.2 and Ruby 3.1.2 +YJIT.

Benchmarks of the home page /

We stress-tested SerpApi's home page, which serves a 75 KB HTML, with 1000 requests and 50 concurrencies. We ran the benchmark with hey -n 1000 "http://localhost:3000/". Here are the results.

 Rails 6.0.3.7Rails 6.1.7Rails 7.0.4 (Ruby 2.7.2)Rails 7.0.4 (Ruby 3.1.2)Rails 7.0.4 (Ruby 3.1.2 +YJIT)
Total14.461714.455314.296614.966812.8496
Slowest3.94664.10333.40274.45724.2234
Fastest0.01730.01640.01520.01640.0134
Average0.63790.62360.63860.6510.5594
Requests/sec69.148169.178969.946566.814477.8231

The results showed that all versions of Rails tested had similar performance on Ruby 2.7.2. Rails 7.0 turned out to be slower on Ruby 3.1.2. However, with YJIT enabled, we saw a significant performance improvement. YJIT is truly making Rails run faster.

We tested SerpApi's most-called API /search, with 100 requests and 20 concurrencies. We lowered the number of requests and concurrency because /search is much slower than the home page, and we didn't want to wait too long. The results were reported by hey -n 100 -c 20 -t 100 "http://localhost:3000/search.json?engine=google&q=coffee&no_cache=true"

 Rails 6.0.3.7Rails 6.1.7Rails 7.0.4 (Ruby 2.7.2)Rails 7.0.4 (Ruby 3.1.2)Rails 7.0.4 (Ruby 3.1.2 +YJIT)
Total70.612967.966868.589269.408175.3803
Slowest16.738817.968817.55616.83818.6639
Fastest7.32826.84127.26317.37316.7363
Average12.765811.986412.416512.614813.8277
Requests/sec1.41621.47131.4581.44081.3266

The results all looked similar except for Ruby 3.1.2 with YJIT enabled, which got worse. This corresponds to the conclusion of Benchmarking Ruby 3.1 (+YJIT) vs. Ruby 2.7 that YJIT does a good job on pure ruby code but performs poorly on codes containing heavy C extensions, specifically Nokogiri in this case.

Conclusion

Rails 6.0, Rails 6.1, and Rails 7.0 do not have noticeable performance differences. We didn't test them extensively, but that was enough for our use case. As a side result, YJIT can be turned on for better performance if the C extension does not take a large part of your code (including the gems used).