Phoenix vs Rails benchmark (2019).

Rails is a great framework. It enables companies to build and launch their products quickly. There are many successful companies that are using Rails to build their stack: Github, Shopify, Basecamp, etc.

Despite being quick with the time-to-market, there’s one thing that Rails fails to deliver: performance.

When building a new product, performance is usually not an issue. Very often new products have no users, or even if they have, it’s just a few, so why bother?

But what if performance is an issue? What if the product is successful and performance really matters? Then Phoenix might be the best option!

Phoenix

Phoenix is a web framework. Similar to Rails, it includes features that enable developers to create complex applications, without having to build everything from scratch.

Phoenix also enables developers to reach a quick time-to-market, but with a huge benefit, as stated in the framework website:

A productive web framework that does not compromise speed or maintainability.

Benchmark

To understand how fast Phoenix is when compared to Rails, I created a simple application using both frameworks. It allowed me to test both frameworks under similar and controlled circumstances.

The performance tests were executed using Siege, an open source regression test and benchmark utility.

I used a total of 4 AWS instances to run this experiment. All instances were launched in the same AWS region. Find the instances specs below:

Purpose Name vCPU Memory (GiB)
Application c5.xlarge 4 8
Application c5.2xlarge 8 16
Application c5.4xlarge 16 32
Siege c5.4xlarge 16 32

Each benchmark session was performed using timed testing, with a fixed time of 10 seconds each.

Every framework and instance combination (e.g. Phoenix / c5.xlarge) was tested using 4 different levels of concurrency (10, 50, 100, 255). Each concurrency level simulates the number of users performing requests to the application at the same time. The higher the concurrency level, the higher is the stress on the application.

Application

The focus of the benchmark is how both Phoenix and Rails behave under similar circumstances. The main goal of the application created for this experiment is to measure the performance of the request/response life-cycle. Some features were removed on purpose, to reduce the number of moving parts throughout the experiments.

How the application works

What’s not part of the application

Command used to start the Rails application

$ RAILS_ENV=production RAILS_MAX_THREADS=5 WEB_CONCURRENCY=[NUM_vCPU] rails s
    

Command used to start the Phoenix application

$ MIX_ENV=prod mix phx.server
    

Results

Below you find charts for four specific values:

Each chart represents a specific instance size (e.g. c5.xlarge, c5.2xlarge and c5.4xlarge). The x-axis is the level of concurrency and the y-axis is the result of the measurement.

In all cases being measured, the higher the value in the y-axis, the faster is the application being subject of the experiment.

Transactions

It represents the number of hits the application received during the testing session. In the specific scenario of this benchmark, it describes how many requests the application was able to handle within the 10 seconds period.



Transaction Rate

It represents the number of transactions that the application was able to handle per second.



Data Transfered

It represents the sum of the data transferred to every concurrent request. It includes both the header and the content of the response. Faster applications are able to handle more requests, consequently transferring more data.



Throughput

It represents the average number of bytes passing through the application every second, to all concurrent requests.

Conclusion

In this experiment, Phoenix has delivered better results than Rails for every single case.

In some scenarios, Phoenix has delivered 15x better performance than Rails. On average Phoenix has shown 7x better performance than Rails, which is quite impressive.

As a next step I want to continue the experiment with the same application and instance types, but go beyond the 255 concurrency level.

If you enjoyed this, you might also like

Elvio Vicosa

Hi, I'm Elvio. I’m an independent software developer consultant based in Berlin, Germany. I’ve spent the last 10+ years working in a wide range of software projects, from internal web dashboards to real-time applications used by millions of people.

More about me