#DockerDays Day 6 – Docker Compose

Until now, we have learned how to create images using dockerfile and used network to create channels for communicating between two containers. But it can be cumbersome to manage each container separately, particularly in multi-container applications which have dozens of containers. Building, running, and connecting multiple containers using individual dockerfile would be pretty hard and consume a lot of time and effort.

So how can we make life simpler when handling multiple containers? The answer lies with docker compose.

Agenda

Introduction to docker compose ?
Advantages of docker compose
Installing Docker Compose
Creating your first Compose file
Important commands

Introduction to Docker-Compose

Docker Compose is a tool for orchestrating containers in multi-container applications. You can use YAML to define your services and use a single command (docker-compose up) to get all your services up and running based on the configurations defined in the docker-compose. YAML is a scripting language, known as Yet Another Markup Language.

Even in a single-container environment, using docker compose allows you to define a tool-independent configuration. For example, volume mounts and port mapping can be defined in the docker-compose.yaml.

Advantages of Docker-Compose

The key advantages of using docker-compose can be outlined as

Portability and Support for CI/CD – The compose file can be shared among the developers easily. This allows developers to recreate the environment easily and also supports an efficient CI/CD pipeline.
Fast and simple configuration : YAML scripts allows developers to easily configure/modify multiple application services from a single place.
Internal Networks : Compose allows to create networks to share data among the different services. These networks cannot be accessed from external sources and hence make the communication network secure.

A simple docker compose file for a single container application could be like the following.

version: "3.4"

services:
  nt.vue:
    image: ${DOCKER_REGISTRY-}vueclient
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8090:80"

Do not worry about syntax. We will go into detail soon. But first, we need to install the docker-compose binary.

Installing Docker Compose

Docker compose requires the docker engine to be installed in your machine. Following the instruction given on the official page for installing docker compose

Official Documentation for installing Docker compose.

You can confirm the installation by running the docker-compose version command which would show the version of the currently installed docker-compose tool

> docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020

Creating your first Compose file

Using the docker-compose involves 3 basic steps.

Define your application’s environment using dockerfile.

We have already familiarized ourselves with dockerfile earlier in this series, we will now proceed to docker-compose file. Create your docker compose file in the root folder as docker-compose.yml. As mentioned earlier, the docker compose file is nothing but an YML file.

For the sake of example, we will create a compose file for vue js application image, which we will build in the previous chapter of this series.

# version: "3.4"

services:
  nt.vue:
    image: ${DOCKER_REGISTRY-}vueclient
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8090:80"

Let us closely examine the above configuration. The first line, version denotes the version of docker compose we are using. This is intentionally commented out because this has been made optional since v1.27.0.

The services provide a list of computer resources your application would consume. These resources can be scaled up/down independent of each other. The services could be created from existing images or can be built as shown in the above example. In the example, a service nt.vue is being defined and created by building the corresponding dockerfile. The service has been configured to use the port configuration 8090:80.

Similarly, you could define the networks and volumes to be used by the service. Let us look into a slightly more complex example now.

services:
  # APPLICATION SERVICES

  # Gateway Service
  nt.gateway:
    image: ${DOCKER_REGISTRY-}ntgateway
    build:
      context: .
      dockerfile: infrastructure/nt.gateway/Dockerfile
    networks:
      services_network:

        # Authentication Service
  authservice.api:
    image: ${DOCKER_REGISTRY-}authserviceapi
    build:
      context: .
      dockerfile: services/AuthService/AuthService.Api/Dockerfile
    networks:
      pg_network:
      mongo_network:
      services_network:

        # User Management Service
  userservice.api:
    image: ${DOCKER_REGISTRY-}userserviceapi
    build:
      context: .
      dockerfile: services/UserService/UserService.Api/Dockerfile
    networks:
      services_network:

        # DATABASE SERVICES

        # Database for Authentication Service
  authdb:
    image: postgres:14.1-alpine
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=ntuserauth
    ports:
      - "5432:5432"
    volumes:
      - auth_data:/var/lib/postgresql/data
    networks:
      pg_network:

        # Auth Service Log Db
  authServiceLog:
    image: mongo:latest
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: password
    networks:
      mongo_network:
    volumes:
      - authServiceLog_Data:/data/db

  # HELPER SERVICES

  # Portainer
  portainer:
    image: portainer/portainer-ce

  # PgAdmin to manage Postgres
  postgres_pgadmin:
    image: dpage/pgadmin4:latest
    environment:
      - PGADMIN_DEFAULT_EMAIL=john.doe@gmail.com
      - PGADMIN_DEFAULT_PASSWORD=password
    ports:
      - "5050:80"
    networks:
      pg_network:

  mongo-express:
    image: mongo-express
    restart: always
    ports:
      - 8081:8081
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: root
      ME_CONFIG_MONGODB_ADMINPASSWORD: admin123
      ME_CONFIG_MONGODB_URL: mongodb://root:admin123@authServiceLog:27017/
    networks:
      mongo_network:

volumes:
  auth_data:
    name: "vol_nt_auth_pg"
  portainer_data:
    name: "vol_nt_portainer"
  authServiceLog_Data:
    name: "vol_nt_auth_log_mongo"

networks:
  pg_network:
    driver: bridge
  mongo_network:
    driver: bridge
  services_network:
    driver: bridge

In the above example, you can notice that there are few volumes and network definitions included. These defined volumes and network are consumed in the services defined. For example, the following defines a mongodb service, which uses the authServiceLog_Data volume and is connected to the mongo_network.

# Auth Service Log Db
authServiceLog:
  image: mongo:latest
  restart: always
  environment:
    MONGO_INITDB_ROOT_USERNAME: root
    MONGO_INITDB_ROOT_PASSWORD: password
  networks:
    mongo_network:
  volumes:
    - authServiceLog_Data:/data/db

Important commands

docker compose up

You can use the docker compose command or directly invoke the docker-compose to initiate or shut down the containers.

>docker-compose up

The docker-compose up command creates and starts the containers and networks specified in the YAML rules of docker file.

By default, the docker-compose up would select the docker-compose.yml file in the current directory. But you could specify a different file (or multiple files) using the -f flag.

> docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d

As mentioned above, docker compose allows multiple isolated environments within a single host. This is achieved by using the project name. By default, the project name is the base name of the directory where the docker-compose.yml file exists.

nt.microservice> docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d
Creating network "ntmicroservice_mongo_network" with driver "bridge"
Creating network "ntmicroservice_pg_network" with driver "bridge"
Creating network "ntmicroservice_default" with the default driver
Creating network "ntmicroservice_services_network" with driver "bridge"
Creating ntmicroservice_mongo-express_1    ... done
Creating ntmicroservice_authServiceLog_1   ... done
Creating ntmicroservice_postgres_pgadmin_1 ... done
Creating ntmicroservice_nt.gateway_1       ... done
Creating ntmicroservice_authdb_1           ... done
Creating portainer                         ... done
Creating ntmicroservice_authservice.api_1  ... done
Creating ntmicroservice_userservice.api_1  ... done

But you could override them using the -p flag and specifying a custom project name. For example,

nt.microservice> docker-compose -p NewProjectName -f docker-compose.yml -f docker-compose.override.yml up -d
Creating network "newprojectname_services_network" with driver "bridge"
Creating network "newprojectname_pg_network" with driver "bridge"
Creating network "newprojectname_mongo_network" with driver "bridge"
Creating network "newprojectname_default" with the default driver
Creating portainer                         ... done
Creating newprojectname_authdb_1           ... done
Creating newprojectname_userservice.api_1  ... done
Creating newprojectname_mongo-express_1    ... done
Creating newprojectname_nt.gateway_1       ... done
Creating newprojectname_authservice.api_1  ... done
Creating newprojectname_authServiceLog_1   ... done
Creating newprojectname_postgres_pgadmin_1 ... done

Using custom project names is particularly useful if you are interested to run multiple versions (branches) of your application suite and compare them, assured that each of the environments is isolated from each other.

docker compose down

To bring down the containers and services defined and created using a docker-compose, you can use the docker compose down command.

> docker compose down
[+] Running 12/12
 - Container ntmicroservice_userservice.api_1   Removed                                                            5.2s
 - Container ntmicroservice_authServiceLog_1    Removed                                                            5.1s
 - Container ntmicroservice_authdb_1            Removed                                                            6.7s
 - Container ntmicroservice_authservice.api_1   Removed                                                            6.7s
 - Container ntmicroservice_nt.gateway_1        Removed                                                            6.7s
 - Container portainer                          Removed                                                            6.7s
 - Container ntmicroservice_mongo-express_1     Removed                                                            3.1s
 - Container ntmicroservice_postgres_pgadmin_1  Removed                                                            6.7s
 - Network ntmicroservice_pg_network            Removed                                                            0.7s
 - Network ntmicroservice_mongo_network         Removed                                                            2.8s
 - Network ntmicroservice_default               Removed                                                            2.1s
 - Network ntmicroservice_services_network      Removed

docker compose pause/unpause

You can temporarily pause all containers of a service using the pause command.

> docker compose pause
[+] Running 8/7
 - Container ntmicroservice_userservice.api_1   Paused                                                             0.0s
 - Container ntmicroservice_authservice.api_1   Paused                                                             0.0s
 - Container ntmicroservice_mongo-express_1     Paused                                                             0.0s
 - Container ntmicroservice_authdb_1            Paused                                                             0.0s
 - Container portainer                          Paused                                                             0.0s
 - Container ntmicroservice_nt.gateway_1        Paused                                                             0.0s
 - Container ntmicroservice_postgres_pgadmin_1  Paused                                                             0.0s
 - Container ntmicroservice_authServiceLog_1    Paused                                                             0.0s

To resume, use the unpause command.

> docker compose unpause

docker compose ps

The docker compose ps lists you all the list of containers within the docker-compose configuration file.

> docker compose ps
NAME                                COMMAND                  SERVICE             STATUS              PORTS
ntmicroservice_authServiceLog_1     "docker-entrypoint.s…"   authServiceLog      running             27017/tcp
ntmicroservice_authdb_1             "docker-entrypoint.s…"   authdb              running             0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
ntmicroservice_authservice.api_1    "dotnet AuthService.…"   authservice.api     running             0.0.0.0:8002->80/tcp, :::8002->80/tcp
ntmicroservice_mongo-express_1      "tini -- /docker-ent…"   mongo-express       running             0.0.0.0:8081->8081/tcp, :::8081->8081/tcp
ntmicroservice_nt.gateway_1         "dotnet nt.gateway.d…"   nt.gateway          running             0.0.0.0:8001->80/tcp, :::8001->80/tcp
ntmicroservice_postgres_pgadmin_1   "/entrypoint.sh"         postgres_pgadmin    running             0.0.0.0:5050->80/tcp, :::5050->80/tcp, 443/tcp
ntmicroservice_userservice.api_1    "dotnet UserService.…"   userservice.api     running             0.0.0.0:8003->80/tcp, :::8003->80/tcp
portainer                           "/portainer"             portainer           running             0.0.0.0:8080->8000/tcp, :::8080->8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 9443/tcp

docker compose images

Similarly, if you are interested in knowing all the images used by the containers, you can use the docker compose images.

> docker compose images
Container                           Repository               Tag                 Image Id            Size
ntmicroservice_authServiceLog_1     mongo                    latest              798d1656acba        698MB
ntmicroservice_authdb_1             postgres                 14.1-alpine         1149d285a5f5        209MB
ntmicroservice_authservice.api_1    authserviceapi           latest              de63106d44e6        241MB
ntmicroservice_mongo-express_1      mongo-express            latest              2d2fb2cabc8f        136MB
ntmicroservice_nt.gateway_1         ntgateway                latest              7e7714ef4150        219MB
ntmicroservice_postgres_pgadmin_1   dpage/pgadmin4           latest              4b5bbddb3624        340MB
ntmicroservice_userservice.api_1    userserviceapi           latest              e7f50421621f        226MB
portainer                           portainer/portainer-ce   latest              ed396c816a75        280MB

Conclusion

In this part of the #dockerdays series, we have familiarized ourselves with the docker compose tool. We will continue exploring docker in this series, and also take time to check support for docker in top tools/IDEs like Visual Studio.