Benchmarking Puma 4 vs. Puma 5 vs. Puma 6

Puma 6.0.0 was released on Oct 19, 2022. We are two major versions behind. Let's benchmark Puma 4 vs. Puma 5 vs. Puma 6 and see how they perform.

We put our focus on testing the throughput of these versions. The excellent HTTP load generator hey made it easy to stress test Puma and get detailed reports.

We benchmarked on SerpApi's home page, which returns a 75 KB HTML with 1000 requests on 50 concurrencies. The tests were run on DigitalOcean CPU-Optimized, 2 vCPUs, 4 GB, Ruby 2.7.2, Rails 6.0.3.7 production mode. Here are the results.

The results showed that the developers are doing great in improving Puma's performance. We can see an increasing number of "Requests/sec" on newer versions.

During the tests, we noticed that compared to Puma 6.0.0, on Puma 4.3.6 and Puma 5.6.5, the two workers were sometimes not evenly loaded, which means they didn't make most of the CPU resources. Looking at the release note, wait_for_less_busy_worker, first introduced in Puma 5, was defaulted in Puma 6. This option resolved the unbalanced workload among workers.

We tried to set the option on Puma 5. As a result, "Requests/sec" increased a bit. However, its performance was still lower than Puma 6.

Migrating to Puma 6 is a no-brainer. But as 6.0.0 has just been released, it would be better to look at its bug reports first. For those who are cautious, upgrading to the latest version of Puma 5 with wait_for_less_busy_worker = 0.005 set is recommended.

That's all for the benchmarks on Puma. Thanks for reading. See you in the following benchmark report.