Blog

Exposing multiple ports in a single container Docker Elastic Beanstalk environment.

2018-11-05

I recently ran into a problem when deploying a new application to Elastic Beanstalk. How to expose multiple ports?

If you deploy to Elastic Beanstalk using the single container Docker environment, each instance will only expose one port; the first one that is specified by the Dockerfile or Dockerrun.aws.json file that you provide with the application.

# only port 8080 will be accessible
EXPOSE 8080 8081

This is clearly stated in the documentation here:

You can specify multiple container ports,
but Elastic Beanstalk uses only the first
one to connect your container to the host's
reverse proxy and route requests from the
public Internet.

This means that we’ll have to customize the nginx configuration to expose a second (or more) ports on each instance.

Searching for a solution to this problem doesn’t help much. Many of the solutions say to use a Multicontainer Docker environment with a single container to specify multiple ports, but this adds an additional layer of abstraction as this particular environment is built around using Amazon’s Elastic Container Service (ECS) in addition to Beanstalk. There are other suggestions related to iptable configuration, but really we need to modify the configuration of nginx.

On shelling into an Elastic Beanstalk EC2 instance and navigating to /etc/nginx/ we find several configuration files, but the ones we are interested in are going to be these

/etc/nginx/
    nginx.conf  <---
    conf.d/
        elasticbeanstalk-nginx-docker-upstream.conf  <---
    sites-enabled/
        elasticbeanstalk-nginx-docker-proxy.conf  <---

Here is the default nginx.conf

# Elastic Beanstalk Nginx Configuration File
user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log;

pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log    /var/log/nginx/access.log;

    log_format  healthd '$msec"$uri"$status"$request_time"$upstream_response_time"$http_x_forwarded_for';

    include       /etc/nginx/conf.d/*.conf;
    include       /etc/nginx/sites-enabled/*;
}

Note that it includes everything in /etc/nginx/conf.d/ and also /etc/nginx/sites-enabled/.

One of the relevant files is /etc/nginx/conf.d/elasticbeanstalk-nginx-docker-upstream.conf. It identifies the instance’s docker container as an upstream, with the first EXPOSE port from the Dockerfile. The server IP is always fixed to the same local IP, as far as I can tell.

upstream docker {
    server 172.17.0.3:8080;
    keepalive 256;
}

The other relevant file is /etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy.conf

    map $http_upgrade $connection_upgrade {
        default        "upgrade";
        ""            "";
    }

    server {
        listen 80;

        gzip on;
            gzip_comp_level 4;
            gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
            set $year $1;
            set $month $2;
            set $day $3;
            set $hour $4;
        }
        access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;

        access_log    /var/log/nginx/access.log;

        location / {
            proxy_pass            http://docker;
            proxy_http_version    1.1;

            proxy_set_header    Connection            $connection_upgrade;
            proxy_set_header    Upgrade                $http_upgrade;
            proxy_set_header    Host                $host;
            proxy_set_header    X-Real-IP            $remote_addr;
            proxy_set_header    X-Forwarded-For        $proxy_add_x_forwarded_for;
        }
    }

This file has most of the proxy pass information for linking the instances external port 80 to the upstream docker container from the elasticbeanstalk-nginx-docker-upstream.conf file.

You can see that Elastic Beanstalk is creating an upstream reference to the Docker container with the first port that is specified in the Dockerfile, and passing it traffic from port 80 externally. So, to open a second port we can create a second upstream that maps our second port, and then provide a server configuration for it that is just like the original with a different port, or with customization if needed.

We can do this easily with an .ebextension called second-port.conf. The file should be stored in the project directory at .ebextensions/second-port.conf and should be deployed with the rest of the application.

It should specify an additional config file for nginx to pick up in the /etc/nginx/conf.d/ directory so that it will be included in the base nginx.conf.

files:
    /etc/nginx/conf.d/second-port.conf:
        mode: "000644"
        owner: root
        group: root
        content: |
            upstream dockertwo {
                server 172.17.0.3:8081;
                keepalive 256;
            }

            server {
                listen 8081;

                gzip on;
                    gzip_comp_level 4;
                    gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

                if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
                    set $year $1;
                    set $month $2;
                    set $day $3;
                    set $hour $4;
                }
                access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;

                access_log    /var/log/nginx/access.log;

                location / {
                    proxy_pass            http://dockertwo;
                    proxy_http_version    1.1;

                    proxy_set_header    Connection            $connection_upgrade;
                    proxy_set_header    Upgrade                $http_upgrade;
                    proxy_set_header    Host                $host;
                    proxy_set_header    X-Real-IP            $remote_addr;
                    proxy_set_header    X-Forwarded-For        $proxy_add_x_forwarded_for;
                }
            }

This file will create the file /etc/nginx/conf.d/second-port.conf on each instance, which opens the specified port to the mapped Docker container port. Also, be sure to map the ports explicitly in the Dockerrun.aws.json file that should also be packaged with your application:

{
  "AWSEBDockerrunVersion": "1",
  "Ports": [
    {
      "ContainerPort": "8080"
    },
    {
      "ContainerPort": "8081",
      "HostPort": "8081"
    }
  ]
}

flynn.gg

Christopher Flynn

Blog

Exposing multiple ports in a single container Docker Elastic Beanstalk environment.

Further reading

Elastic Beanstalk

Docker

nginx