Making 12factor Elixir/Phoenix releases

Elixir had a bad reputation for its deployment story due to the complex tooling and compile-time configuration preference. That is history now as we can easily make Elixir v1.11 releases with the runtime configuration to adhere to the 12factor style of deployment.

If you don’t know what 12factor is, it’s a document made at Heroku with recommendations how to design your applications. Although the purpose was most likely about stirring people into making applications that would run smoothly on the Heroku platform, it’s a quite sensible set of recommendations.

I don’t think you have to adhere to 12factor at all costs, but some points make sense. This post is namely about section III., which recommends storing configuration in an environment. Something a bit problematic in Elixir before, but something I always wanted.

Sections on dependencies and logs are also relevant, while sections on stateless processes and concurrency might not apply to us as Beam has its own lightweight stateful processes. However, you can decide to keep Elixir nodes stateless and use something like Redis.

But back to the runtime configuration. Suppose we design our application to depend only on the runtime environment variables. In that case, we can make true release artifacts that can be used across organizations and workloads (well, almost, as we don’t have a streamlined way of cross-compiling Elixir). Now compare it to the compile-time configuration, which bakes in your database credentials.

Let’s imagine a minimal Phoenix application. It will come with some config files in the config/ directory. Namely a common config/config.exs and config/prod.exs for configuring production release. Those files provide the build-time application configuration to use values it has available when building the release.

Your Elixir application might combine the compile and runtime configuration in the end, but the following is about providing all your ENVs at runtime.

The first question I had was, “what’s the minimal amount of work to make a Phoenix release?”

It seems we can just call to Mix with the right MIX_ENV environment:

$ MIX_ENV=prod mix release new_phoenix

The new_phoenix is just a name. You can omit it, and your project name will be used instead.

We’ll get the following output:

Release created at _build/prod/rel/new_phoenix!

    # To start your system
    _build/prod/rel/new_phoenix/bin/new_phoenix start

Once the release is running:

    # To connect to it remotely
    _build/prod/rel/new_phoenix/bin/new_phoenix remote

    # To stop it gracefully (you may also send SIGINT/SIGTERM)
    _build/prod/rel/new_phoenix/bin/new_phoenix stop

To list all commands:

    _build/prod/rel/new_phoenix/bin/new_phoenix

This was easy! Let’s start it:

$ _build/prod/rel/new_phoenix/bin/new_phoenix start

It seems to be working, but how do we access the application? The port 4000 is not working as usual.

Chances are that you are missing server: true in your endpoint configuration:

# config/prod.exs
config :new_phoenix, NewPhoenixWeb.Endpoint,
  http: [:inet6, port: System.get_env("PORT") || 4000],
  ...
  server: true

Reruning mix release will now start the server at port 4000. It’s now a fully self-contained release (usable on the same architecture) and it will also log to STDOUT by default. That’s two things done from the 12factor checklist.

What if we want to change the PORT at runtime? Maybe some servers will already use port 4000 for something else. But we don’t know that beforehand to provide it while building the release!

We have to move this configuration to runtime. Thanks to the recently added config/runtime.exs support (replacing deprecated config/releases.exs), we can do it without a sweat:

# config/runtime.exs
import Config

if config_env() == :prod do
  config :new_phoenix, NewPhoenixWeb.Endpoint,
    http: [:inet6, port: System.fetch_env!("PORT")],
    ...
    server: true
end

The only difference at this point is the name of the file. I also changed System.get_env/ to System.fetch_env!/1, which will fail to start the application if the port is not provided. That’s something you might want to do when it comes to things like databases.

Note that config/runtime.exs is evaluated in all environments, therefore we provide the configuration only for the :prod environment.

Rerunning works as expected:

$ MIX_ENV=prod mix release new_phoenix
$ PORT=4000 build/prod/rel/new_phoenix/bin/new_phoenix start

While I added just one configurable variable, you can move all your configuration to runtime this way. And you can forget about the config/prod.secret.exs file. I think this is simpler in the long run and can match how you deploy your Ruby, Python, and other stacks workloads.

Don’t forget most project will need SECRET_KEY_BASE which you can always generate with:

$ mix phx.gen.secret

Again, before we needed it already for the release, but today we can make it completely a run-time dependency.

And then, full-stack applications will have to handle the front-end concerns as well. If you have cache_static_manifest as part of your configuration, run mix phx.digest first to build the static files in priv/static for the release:

$ mix deps.get --only prod
$ MIX_ENV=prod mix compile
$ npm install --prefix ./assets
$ npm run deploy --prefix ./assets
$ MIX_ENV=prod mix phx.digest
$ MIX_ENV=prod mix release new_phoenix
$ PORT=4000 build/prod/rel/new_phoenix/bin/new_phoenix start

If you want to use this release on a production server, copy it over with something like scp and run it from there.

You can run it in the daemon mode, or ideally with a system process manager like systemd. Here are the list of subcommands for the release executable:

The known commands are:

    start          Starts the system
    start_iex      Starts the system with IEx attached
    daemon         Starts the system as a daemon
    daemon_iex     Starts the system as a daemon with IEx attached
    eval "EXPR"    Executes the given expression on a new, non-booted system
    rpc "EXPR"     Executes the given expression remotely on the running system
    remote         Connects to the running system via a remote shell
    restart        Restarts the running system via a remote command
    stop           Stops the running system via a remote command
    pid            Prints the OS PID of the running system via a remote command
    version        Prints the release name and version to be booted

I will look into proper systemd configuration for the release in one of the upcoming post.

← BUY THE PRE-RELEASE

I am writing a complete guide on web application deployment. Ruby with Puma, Python with Gunicorn, NGINX, PostgreSQL, Redis, networking, processes, systemd, backups, and all your usual suspects. 19 chapters and 2 case studies already available.

More →