Out-of-band work and out-of-band garbage collection

Relevant selection for this article:

The Out-of-Band Work feature allows you to run arbitrary long-running tasks outside normal request cycles. This works by letting current requests to the process finish, then telling the process to perform the out-of-band work, then resuming passing requests to the process after said work is finished.

Table of contents

1. Loading...

Out-of-band garbage collection

A specific (and perhaps primary) use case of of Out-of-Band Work is Out-of-Band Garbage Collection. The garbage collector is run outside normal request cycles so that garbage collection runs inside normal request cycles can finish a lot faster. This is especially useful on Ruby versions prior to 2.2, which have less efficient garbage collectors.

Since Ruby version 2.2, out-of-band garbage collection is no longer needed, but on older Ruby versions out-of-band garbage collection can potentially save tens to hundreds of milliseconds of latency in requests.

Properties

Because Out-of-Band Work is implemented at the Passenger inter-process request routing level, and not by – say – spawning a thread inside the application process, Out-of-Band Work has the following useful properties:

• It works well even with tasks that can pause all threads. The MRI Ruby garbage collector is a stop-the-world mark-and-sweep garbage collector.
• Passenger can spawn more processes as necessary, in order to prevent situations in which all application processes are busy performing out-of-band work. Passenger guarantees that there's at least one process that's ready to process requests.
• Passenger guarantees that no more than 1 process is performing out-of-band work at the same time.

Usage

Applications can use Out-of-Band Work as follows:

1. Ensure that the max pool size and min instances options are both larger than 1. Out-of-band work only works if there are at least 2 application processes.
2. Request out-of-band work by outputting the !~Request-OOB-Work header during a request. It does not matter what the value is. At this time, it is not possible to request out-of-band work from outside requests.
3. You can actually perform out-of-band work when you receive a :oob_work Passenger event.

Note that even though you can request out-of-band work, there's no guarantee that Passenger will send an oob_work event in a timely manner, if at all. It is also possible that Passenger sends an oob_work event without you ever having requested one. This latter could for example happen if the OOB work is administrator-initiated. Do not make any assumptions in your code.

Here's an example which implements out-of-band garbage collection using the Out-of-Band framework. This example code doesn't do anything when the code is not being run in Passenger, thanks to the if block.

# Somewhere in a controller method:
# Tell Passenger we want to perform OOB work.
response.headers["!~Request-OOB-Work"] = "true"

# Somewhere during application initialization:
if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:oob_work) do
    # Passenger has told us that we're ready to perform OOB work.
    t0 = Time.now
    GC.start
    Rails.logger.info "Out-Of-Bound GC finished in #{Time.now - t0} sec"
  end
end

For your convenience, Passenger provides a Rack middleware for out-of-band garbage collection. Add the following to your config.ru. Likewise, this example code doesn't do anything when the code is not being run in Passenger, thanks to the if block.

if defined?(PhusionPassenger)
  PhusionPassenger.require_passenger_lib 'rack/out_of_band_gc'

  # Trigger out-of-band GC every 5 requests.
  use PhusionPassenger::Rack::OutOfBandGc, 5
end

It should be noted that, although the application uses the Passenger Ruby API, it is not necessary to add Passenger to the Gemfile.

Further reading

The Phusion Blog article which first introduced this feature.