Phusion Passenger Design and Architecture

Here are some common models that are in use:

The above models cover how nearly all web applications work, whether they’re based on PHP, Django, J2EE, ASP.NET, Ruby on Rails, or whatever. Note that all of these models provide the same functionality, i.e. no model can do something that a different model can’t. The critical reader will notice that all of these models are identical to the one described in the first diagram, if the combination of web servers, application servers, web applications etc. are considered to be a single entity; a black box if you will.

It should also be noted that these models do not enforce any particular I/O processing implementation. The web servers, application servers, web applications, etc. could process I/O serially (i.e. one request at a time), could multiplex I/O with a single thread (e.g. by using select(2) or poll(2)) or it could process I/O with multiple threads and/or multiple processes. It depends on the implementation.

Of course, there are many variations possible. For example, load balancers could be used. But that is outside the scope of this document.

As you’ve seen, administrators often put the web application or its application server behind a real web server in a reverse proxy setup, even when the web app/app server already speaks HTTP. This is because implementing HTTP in a proper, secure way involves more than just speaking the protocol. The public Internet is a hostile environment where clients can send any arbitrary data and can exhibit any arbitrary I/O patterns. If you don’t properly implement I/O handling, then you could open yourself either to parser vulnerabilities, or denial-of-service attacks.

Web servers like Apache and Nginx have already implemented world-class I/O and connection handling code and it would be a waste to reinvent their wheel. In the end, putting the application in a reverse proxying setup often makes the whole system more robust and and more secure. This is the reason why it’s considered good practice.

A typical problem involves dealing with slow clients. These clients may send HTTP requests slowly and read HTTP responses slowly, perhaps taking many seconds to complete their work. A naive single-threaded HTTP server implementation that reads an HTTP requests, processes, and sends the HTTP response in a loop may end up spending so much time waiting for I/O that spends very little time doing actual work. Worse: suppose that the client is malicious, just leaves the socket open and never reads the HTTP response, then the server will spend forever waiting for the client, not being able to handle any more requests. A real-world attack based on this principle is Slowloris.

There are many ways to solve this problem. One could use one thread per client, one could implement I/O timeouts, one could use an evented I/O architecture, one could have a dedicated I/O thread or process buffer requests and responses. The point is, implementing all this properly is non-trivial. Instead of reimplementing these over and over in each application server, it’s better to let a real web server deal with all the details and let the application server and the web application do what they’re best at: their own core business logic.

When an HTTP client sends a request, it is received by the web server (Nginx or Apache). Both Apache and Nginx can be extended with modules. Phusion Passenger provides such a module. The module is loaded into Nginx/Apache. It checks whether the request should be handled by a Phusion Passenger-served web application, and if so, forwards the request to the Passenger core. The internal wire protocol used during this forwarding, is a modified version of SCGI.

The Nginx module and Apache module have an entirely different code base. Their code bases are in ext/nginx and ext/apache2, respectively. Both modules are relatively small because they outsource most logic to the Passenger core, and because they utilize a common library (ext/common). This allows us to support both Nginx and Apache without having to write a lot of things twice.

The Passenger core is where most of the processing is done. The core keeps track of which application processes currently exist, and using load balancing rules, determines which process a request should be forwarded to. The core also takes care of application spawning: if it determines that having more application processes is necessary or beneficial, then it will make that happen. Process spawning is subject to user-configured limits: the core will never spawn more processes than a user-configured maximum.

The core also has monitoring and statistics gathering capabilities. It constantly keeps track of applications' memory usage, how many requests they’ve handled, etc. This information can later be queried from administration tools. And if an application process crashes, the core restarts it.

The core is by far the largest and most complex part of the system, but it is itself composed of several smaller subsystems. Most of the core architecture chapter is devoted to describing the core.

The core cooperates with the UstRouter. This latter is responsible for sending data to Union Station, a monitoring web service. If you didn’t explicitly tell Phusion Passenger to send data to Union Station, then the UstRouter sits idle and does not consume resources.

The core and the UstRouter contain complex logic, so they could contain bugs which could crash them. So as a safety measure, they are both monitored by the Watchdog. If either of them crash, they are restarted by the Watchdog. This setup seeks to ensure that the system stays up, no matter what.

You might now wonder: what happens if the Watchdog crashes? Shouldn’t the Watchdog be monitored by another Watchdog? We’ve contemplated this possibility, but the Watchdog is very simple, and since 2012 we haven’t seen a single report of the Watchdog crashing, nor have we been able to make it crash since that time. So, for the sake of keeping the codebase as simple as possible, we’ve chosen not to introduce multiple Watchdogs.

Finally, there is an array of command line tools which support Phusion Passenger. The installers — passenger-install-*-module — are responsible for installing Phusion Passenger. There are administrative tools such as passenger-status and passenger-memory-stats. And many more. Some of these tools may communicate with one of the agents. For example, the passenger-status queries the core for information that the core has collected. How this communication is done, is described in Instance state and communication.

You might have noticed that Phusion Passenger Standalone is not part of the diagram. So how does it fit into the architecture? Well, Phusion Passenger Standalone is actually just Phusion Passenger for Nginx. The passenger start command simply sets up a modified and stripped-down Nginx web server (which we call the WebHelper) with the Phusion Passenger Nginx module loaded.

The web server creates a new connection to the core on every request. Thus, from the viewpoint of the RequestHandler, its client is the web server. Every time a client connects (i.e. a new request is forwarded), the RequestHandler creates a new Client object which represents that request. All request-specific state is stored inside the Client. After the RequestHandler is done processing a request, it closes the client socket.

Note that in the diagram, a Client has a 0..1 association with RequestHandler. That’s because when a Client is disconnected, the pointer to the associated RequestHandler is set to NULL. There might be background operations left which still have a pointer to the Client. As soon as those background operations finish, they check whether the Client has a valid pointer to the RequestHandler. If so, they commit their work; if not, they discard their work. The Client is destroyed when all its associated background operations have finished.

The RequestHandler asynchronously asks the ApplicationPool subsystem to select an appropriate application process to handle this request. The ApplicationPool checks whether there is an appropriate process, and if not, tries to spawn one. Maybe spawning is not possible right now because of configured resource limits, and we have to wait. In any case, the ApplicationPool takes care of all the nasty details and book keeping, and eventually replies back to the RequestHandler with either a Session object, or an exception.

A Session object represents a single request/response cycle with a particular application process. The RequestHandler uses the information in this Session object to establish a connection with that process and forwards the request, using a protocol that the application prefers and that the RequestHandler supports. The process performs work, and replies back with an HTTP response. The RequestHandler parses and postprocesses the response, and sends a response back to the web server.

If the ApplicationPool replied with an exception, the RequestHandler sends back an error response.

The RequestHandler uses the evented I/O model. This means that the RequestHandler handles many clients (requests) at the same time, using a single thread, inside a single process. This is possible through the use of I/O event multiplexing mechanisms, which are provided by the OS. Examples of such mechanisms include the select(), poll(), epoll() and kqueue() system calls. But those mechanisms are very low-level and OS-specific, so the RequestHandler uses two libraries which abstract away the differences and provide a higher-level API: libev and libuv.

The evented I/O model is also used in Nginx. It is in contrast to the single-threaded multi-process model which handles 1 client per process (used by Apache with the prefork MPM), or the multi-threaded model which handles 1 client per thread (used by Apache with the worker MPM). You can learn more about evented I/O and the different I/O models through these resources:

Spawning begins when the spawn() method is called on a Spawner object. The Spawner determines which user the process should run as, and sets up some communication channels (anonymous Unix domain socket pairs), and forks a process. The parent waits until the child exits, or replies with something over the communication channel.

The communication channel in question is — from the viewpoint of the (pre)loader — actually just stdin, stdout and stderr! The anonymous Unix domain socket pairs that the Spawner creates, is mapped to the child process’s stdin, stdout and stderr file descriptors. Thus, Spawner sends data to the (pre)loader by writing stuff to its stdin, and the (pre)loader sends data back to the Spawner by writing stuff to stdout or stderr.

Because the Passenger core is heavily multi-threaded, the child process has been forked by the Spawner may only perform async-signal-safe operations:

Don’t worry if you don’t know what this means. The point is, there’s almost nothing the forked process can safely do at that stage. So it outsources most of the remaining preparation work to an external executable, the SpawnPreparer. The SpawnPreparer starts with a clean environment where it can safely execute code.

To execute the SpawnPreparer, the child process executes one of the following commands:

How path-to-loader-or-preloader is determined, is described in The AppTypes registry.

The SpawnPreparer is responsible for setting up certain environment variables, current working directory, and other process environmental conditions. When SpawnPreparer is done, it executes the loader or the preloader.

If passenger_spawn_method is set to smart (the default), and there is a preloader available for the application’s programming language, then this step executes the language-specific preloader. If either of the previous conditions are not met (and thus the passenger_spawn_method is automatically forced to direct), then this step executes the language-specific loader.

All (pre)loaders are located in the helper-scripts directory in the source tree. Here are some of the (pre)loaders that are used:

The AppTypes registry keeps a list of available (pre)loaders, and which languages they belong to.

What the loader does is described in Loaders. Likewise, preloaders are described in Preloaders.

The handshaking process begins with a protocol version handshake. The loader printing the line !> I have control 1.0. The Spawner then sends "You have control 1.0", which the loader checks. If the loader observes that the version handshake does not match the expectation, then it aborts with an error.

The Spawner also sends a list of key-value pairs, which is terminated by an empty newline. Upon receiving the empty newline, the Spawner proceeds with loading the application.

Example:

How the application is loaded, depends on the programming language. Here are some examples:

If no errors occur, the loader proceeds with setting up a server. Otherwise, it reports an error.

The loader sets up a server on which the application listens for requests. The Spawner doesn’t care how this is done, how this server works, or even what its concurrency is. It only cares about how it can contact the server. So the loader has full freedom in this step.

As explained in section Request handling and subsection Forwarding to the application, the RequestHandler can talk with the application process in a protocol that the application prefers. The RequestHandler supports two protocols:

Typically, the server is setup to listen on a Unix domain socket file, inside the backends subdirectory of the generation directory. The path to the generation directory was passed during handshake. However the server may also listen on a TCP socket.

Once a server has been setup, the loader can report readiness.

Once the server is set up, the loader sends back a !> Ready response, followed by information about where the server socket listens on, and what protocol it expects. The response is terminated with a `!> ` line (notice the trailing whitespace, which is required).

The information about where the server socket listens on, is a 4-tuple:

Here’s an example of what the Node.js loader sends as response:

After reporting readiness, the loader can enter a main loop and wait for termination.

If something goes wrong in any of the stages, the loader can report an error in two ways:

The loader’s main loop’s job is to wait until a single byte has been received on stdin. As long as the byte has not been received, the loader should not exit, and should keep processing requests. When the byte has been received, the following conditions are guaranteed to be true:

This guarantee is enforced by the RequestHandler and the ApplicationPool. Thus, the loader doesn’t have to perform any kind of complicated shutdown. It can just exit the process.

If the server was listening on a Unix domain socket file, then the loader doesn’t even have to remove the file. The ApplicationPool already takes care of that.

All lines that the loader writes to stdout, and that are not prefixed with `!> `, are forwarded by the Passenger core to its own stdout. Similarly, everything that the loader writes to stderr, whether prefixed with `!> ` or not, is forwarded by the Passenger core to its own stdout.

While the Spawner is still doing its work, it takes care of this forwarding by itself. Once the Spawner is done, it outsources this work to two PipeWatcher objects, each which spawns a background thread for this purpose.