Concurrency and parallelism in Ruby

Ruby

Published on by

Mathieu EUSTACHY

Mathieu EUSTACHY

7 minutes reading

Last article we talked about processes in Ruby and how they are structured. Today we will talk about concurrency and parallelism in Ruby and its ecosystem, what are these 2 concepts and why are they so important to understand.


This will allow us to introduce in the next article the thread notion, after both of these articles you will be able to understand what are workers, what are threads, how they work together and how to set them properly. Moreover, you will understand each line of the below code present by default in a Rails 7 app does:


# config/puma.rb

# Puma can serve each request in a thread from an internal thread pool.
# The `threads` method setting takes two numbers: a minimum and maximum.
# Any libraries that use thread pools should be configured to match
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count


# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together
# the concurrency of the application would be max `threads` * `workers`.
# Workers do not work on JRuby or Windows (both of which do not support
# processes).
#
workers ENV.fetch("WEB_CONCURRENCY") { 2 }


(Yes, you will understand it, trust me an read this article and the next one till the end)


Let’s dive into it!


This article is the 3rd article of a broader series about “low-level” computing concepts applied to Ruby.

  1. What is a Ruby implementation?
  2. Process management in Ruby
  3. Concurrency and parallelism in Ruby
  4. Thread management in Ruby
  5. Memory management in Ruby


Always keep in mind that I am intentionally summarising things to give you a quick overview, there is more to each concept.


Concurrency & parallelism


Let’s start by defining concurrency and parallelism which are the underlying concepts explaining this code.


Concurrency can be defined as the ability of a system to execute multiple tasks in overlapping time periods.

Whereas parallelism can be defined as the simultaneous execution of multiple tasks, where each task runs independently. Tasks execute at the same time, taking advantage of multiple processors or cores.


Below are two schemas to illustrate concurrency vs parallelism:



Now, you might believe, as I did at first, that parallelism is always better than concurrency. However, you should asses which one to use with two crucial factors in mind:


A. Nature of Tasks


When a task is largely independent and don't heavily rely on shared resources or data (like database data), parallelism can be a really good choice, it allows multiple tasks to progress simultaneously by leveraging the full potential of the available computing power. This is particularly true for CPU-bound tasks (ie tasks that do a lot of calculations) which can be divided into independent subtasks.


On the other hand, when tasks are interdependent (ie its output influences another task), or if it involves more I/O-bound operations (I/O stands for Input/Output, like database calls for example), concurrency may be a more appropriate choice. In this scenario, during the I/O wait (like waiting for a database query to return), through concurrency, the application can switch to another task until the I/O answer.


B. Computing Resources


The availability of computing resources also plays a crucial role in deciding between concurrency and parallelism.


In general, parallelism tends to consume more computing power resources resources than concurrency for a lot of reasons. To put it simply, when multiple tasks run independently, there's a need for additional management, which can require more computational power (I cannot develop more on that here, it has to be its own article).


So, when computing resources are limited, concurrency is often the most pragmatic choice. But that does not necessarily mean that concurrency is cheap or compromise option at all: a system designed for effective concurrency management can provide as great responsiveness and handle multiple tasks as efficiently as a system designed for parallelism.


To summarise it:



Concurrency and parallelism in Ruby


Now that we have explained the 2 main decision factors, let’s see how concurrency and parallelism come into play when processing heavyweight tasks in Ruby.


As a student at @LeWagon, I remembered when we first talked about background jobs, it was explained as a way to process heavyweight tasks without blocking the whole application. For example, if you have to make a call to an external API, to upload a file, or send an email, it is the perfect use case for a background job. This task is time-consuming and should not block the whole application while being executed.


Hopefully, thanks to Ruby on Rails magic, you just had to declare an ApplicationJob class, initialise a new instance of this class in your code and that’s it! Rails was taking care of the necessary lifting for our application to perform well using concurrency (using Sidekiq) or parallelism (using Resque)! This way the application was still running smoothly for our users while also completing some heavyweight tasks at the same time.


For sure, if you are in the Ruby on Rails ecosystem, you have heard of Sidekiq, it’s the go-to queuing system. But why? Why is a concurrent queuing system the go-to option in Rails?


It’s because it’s commonly admitted that a significant portion of time in Ruby background jobs is spent waiting on … I/O-bound operations (database query, API calls, …), therefore leveraging concurrency in background jobs is the way to go.


Yes, and you can see the power of concurrency with the below benchmark displayed on Sidekiq Github repository:



It only takes 23.5s to perform 500k lightweight jobs on a single CPU 🙃. This should be a good starting point for your Rails app.



Understanding the introduction code in puma.rb


Before finishing this article, I’ll finish by explaining the puma.rb code we mentioned earlier.


When you execute bundle exec rails s in your terminal, you will most likely launch a Puma server since it’s the default web server of Rails. Puma uses config/puma.rb for its configuration (the file we mentioned in the introduction):



Circled in green, you have the number of threads (= execution units, more on that in the next article) which relates to concurrency in the Ruby world (MRI Ruby world to be precise) . These numbers are defined by these lines in config/puma.rb:

# puma.rb
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count

The bigger the value the more concurrent your app is.


Circled in purple, you have the number of workers which relates to parallelism in the Ruby world (again, MRI Ruby world to be precise). This number is defined by this line in config/puma.rb:

# puma.rb
workers ENV.fetch("WEB_CONCURRENCY") { 2 }

The bigger the WEB_CONCURRENCY value the more parallel your app is.


As you can observe, the terms "concurrency" and "parallelism" are often used interchangeably, but, as we previously saw, they refer to slightly different concepts in the context of programming. Hence, when talking about Puma workers and WEB_CONCURRENCY, we are instead talking about parallelism (😅).


We can therefore conclude that Puma web server can use concurrency and parallelism at the same time. But in most config/puma.rb files, the workers ENV.fetch("WEB_CONCURRENCY") { 2 } line is commented out, therefore there is only one process running several threads, making it an only concurrent web server.



Wrapping it up


In the Ruby world (again, MRI Ruby world to be precise), parallelism is closely related to a process (remember last article), concurrency is closely related to a thread (more on that in the next article).


But more on threads and workers in the next article of this series. It will complete this one by diving into thread management in Ruby, and it will show in more depth how parallelism and concurrency are used in Ruby and its ecosystem. Get ready for it!



Resources used:



Back to articles

My last articles

In depth Ruby memory optimisation: `true`, `false` and `nil`

Ruby

Published on by

Mathieu EUSTACHY

Mathieu EUSTACHY

Memory management explained in Ruby

Ruby

Published on by

Mathieu EUSTACHY

Mathieu EUSTACHY

Thread management in Ruby

Ruby

Published on by

Mathieu EUSTACHY

Mathieu EUSTACHY