All.docker.tutorial

Docker#

Introduction#

Docker is an open-source tool designed to make our life typically easier (although it takes energy and time to master) when creating, building, deploying, and running software applications.
Docker can package an application and its dependencies in a virtual container that can run on any Linux, Windows, or macOS computer.
Our Docker containers have everything required (e.g. OS packages, Python packages) inside to run certain applications/code.

Concepts#

Docker image#

A Docker image is a read-only template with instructions for creating a Docker container
Typically a Docker image includes needed libraries and packages and their versions

Dockerfile#

A Dockerfile is a text document that contains all the commands to call on the command line to assemble an image. E.g. //cmamp/devops/docker_build/dev.Dockerfile.

Docker container#

A Docker container is a runnable instance of an image. One can run code inside a Docker container having all requirements installed.

Docker registry#

A Docker registry stores Docker images. In other words, Docker registry for docker images is like GitHub for code.

Poetry#

Poetry is a tool for managing Python packages and dependencies and allows to:
- List packages you want to install with some constraints
  - E.g., pandas must be above 1.0 in devops/docker_build/pyproject.toml
- Given a list of packages you need to install to get the desired environment, poetry "optimizes" the package versions and generate devops/docker_build/poetry.lock, which contains the list of versions of the packages to install
- If there is a new version of a package re-running poetry might give you an updated list of packages to install

Build a Docker image#

General#

A docker image is built from a Dockerfile. The image is then used to run a Docker container.
There is /devops dir under a project's dir that contains Docker-related files, e.g. cmamp/devops

Base image#

A Dockerfile should start with specifying a base image.
The base image is an image that a new image is built on top of. A new Docker image will have all the packages/dependencies that are installed in the base image.
Use FROM statement to specify a base image, e.g.
```
FROM ubuntu:20.4
```

Copy files#

Copy files that are required to build a Docker image to the Docker filesystem
To copy a file from /source_dir (your filesystem) to /dst_dir (Docker filesystem) do:
```
COPY source_dir/file dst_dir
```
E.g., the command below will copy install_packages.sh from devops/docker_build to the Docker's root directory so that install_packages.sh can be accessed by Docker
```
COPY devops/docker_build/install_packages.sh .
```

Install OS packages#

Install OS packages that are needed for a Docker app, but that are not installed for a base image
Use RUN instruction to install a package, e.g.
```
RUN apt-get install postgresql-client
```
Alternatively you can package all installation instructions in a .sh file and run it. Do not forget to copy a .sh file to the Docker filesystem so that Docker can see it. E.g.,
```
COPY devops/docker_build/install_packages.sh .
RUN /bin/sh -c "./install_packages.sh"
```

Install Python packages#

We prefer to install Python packages with poetry
Make sure that there is instruction to install pip3 and poetry. You can either put it in a Dockerfile or in a separate file like install_packages.sh.
```
RUN apt-get install python3-pip
RUN pip3 install poetry
```
Copy poetry-related files to the Docker filesystem so that files can be accessed by Docker
```
COPY devops/docker_build/poetry.toml
COPY devops/docker_build/poetry.lock
```
Install Python packages
```
RUN poetry install
```

Build an image from a Dockerfile#

To build an image from a Dockerfile run:
```
> docker build .
```
The Dockerfile must be called Dockerfile and located in the root of the build context
You can point to any Dockerfile by using -f:
```
> docker build -f /path/to/dockerfile
```

Run multi-container Docker application#

Docker Compose is a tool for defining and running multi-container Docker applications
With Docker Compose you use a YAML file to configure your application's services

Version#

At the beginning of a docker-compose.yaml file specify the docker-compose version. For more information see the official documents
```
version: "3.0"
```

Images#

You can either re-use a public image or build a new one from a Dockerfile

The app service below uses the image that is built from the dev.Dockerfile

app:
    build:
        context: .
        dockerfile: dev.Dockerfile

The im_postgres_local service below uses the public postgres image pulled from the Docker hub registry
```
im_postgres_local:
    image: postgres: 13
```

Bind mount#

If you want to be able to share files between the host and a Docker container, you should bind-mount a directory
E.g. mount current directory to /app dir inside a Docker container:
```
app:
    volumes:
        - .:/app
```

Environment variables#

You can either use variables directly from the environment or pass them in a docker-compose.yaml file
It is supposed that POSTGRES_VERSION is already defined in the shell.
```
db:
    image: "postgres:${POSTGRES_VERSION}"
```

Set environment variable in a service's container

db:
    environment:
        - POSTGRES_VERSION=13
        image: "postgres:${POSTGRES_VERSION}"

Set environment variable with .env file

db:
    env_file:
        - ./postgres_env.env
        image: "postgres:${POSTGRES_VERSION}"

File postgres_env.env

> cat ./postgres_env.env
POSTGRES_VERSION=13

Basic commands#

To check more advanced usage, please see the official documentation.

Let's assume that the docker-compose.yaml file is located in the current dir

# Build, (re)create, start, and attach to containers for a service.
> docker-compose up

# List containers
> docker-compose ps

# Stop containers created with `up`
> docker-compose down