Benchmarking Rails 6 vs. Rails 7
This blog post is a continued work of Benchmarking Ruby 3.1 (+YJIT) vs. Ruby 2.7. We wanted to check the performance of Rails 6.1 and Rails 7.0 compared to Rails 6.0, which we are currently on.
With the help of hey, we used a similar approach to benchmarking Puma as the last time.
We ran all benchmarks on DigitalOcean CPU-Optimized, 2 vCPUs, 4 GB, Ruby 2.7.2, and Rails, all booted in production mode. As Rails 7 prefers Ruby 3+, we tested Rails 7.0 on Ruby 3.1.2 and Ruby 3.1.2 +YJIT.
Benchmarks of the home page /
We stress-tested SerpApi's home page, which serves a 75 KB HTML, with 1000 requests and 50 concurrencies. We ran the benchmark with hey -n 1000 "http://localhost:3000/". Here are the results.
| Rails 6.0.3.7 | Rails 6.1.7 | Rails 7.0.4 (Ruby 2.7.2) | Rails 7.0.4 (Ruby 3.1.2) | Rails 7.0.4 (Ruby 3.1.2 +YJIT) | |
|---|---|---|---|---|---|
| Total | 14.4617 | 14.4553 | 14.2966 | 14.9668 | 12.8496 |
| Slowest | 3.9466 | 4.1033 | 3.4027 | 4.4572 | 4.2234 |
| Fastest | 0.0173 | 0.0164 | 0.0152 | 0.0164 | 0.0134 |
| Average | 0.6379 | 0.6236 | 0.6386 | 0.651 | 0.5594 |
| Requests/sec | 69.1481 | 69.1789 | 69.9465 | 66.8144 | 77.8231 |
The results showed that all versions of Rails tested had similar performance on Ruby 2.7.2. Rails 7.0 turned out to be slower on Ruby 3.1.2. However, with YJIT enabled, we saw a significant performance improvement. YJIT is truly making Rails run faster.
Benchmarks of /search
We tested SerpApi's most-called API /search, with 100 requests and 20 concurrencies. We lowered the number of requests and concurrency because /search is much slower than the home page, and we didn't want to wait too long. The results were reported by hey -n 100 -c 20 -t 100 "http://localhost:3000/search.json?engine=google&q=coffee&no_cache=true"
| Rails 6.0.3.7 | Rails 6.1.7 | Rails 7.0.4 (Ruby 2.7.2) | Rails 7.0.4 (Ruby 3.1.2) | Rails 7.0.4 (Ruby 3.1.2 +YJIT) | |
|---|---|---|---|---|---|
| Total | 70.6129 | 67.9668 | 68.5892 | 69.4081 | 75.3803 |
| Slowest | 16.7388 | 17.9688 | 17.556 | 16.838 | 18.6639 |
| Fastest | 7.3282 | 6.8412 | 7.2631 | 7.3731 | 6.7363 |
| Average | 12.7658 | 11.9864 | 12.4165 | 12.6148 | 13.8277 |
| Requests/sec | 1.4162 | 1.4713 | 1.458 | 1.4408 | 1.3266 |
The results all looked similar except for Ruby 3.1.2 with YJIT enabled, which got worse. This corresponds to the conclusion of Benchmarking Ruby 3.1 (+YJIT) vs. Ruby 2.7 that YJIT does a good job on pure ruby code but performs poorly on codes containing heavy C extensions, specifically Nokogiri in this case.
Conclusion
Rails 6.0, Rails 6.1, and Rails 7.0 do not have noticeable performance differences. We didn't test them extensively, but that was enough for our use case. As a side result, YJIT can be turned on for better performance if the C extension does not take a large part of your code (including the gems used).