A Docker Introduction

---

---

---

---

# Docker - Tips and Best Practices

.align-center.v-align-bottom.no-bullets.small[
- presentation by: Tiago Carreira
]

]

---

# About me

Operations Engineer (DevOps) @ SAPO
 
.small[
- MSc in Electrical Engineering
- working with Linux for 6y+
- favourite prog languages: bash, python, go
- favourite text editor: vscode, vim
- music, sports, therapeutic massages, bad jokes
- .tiny[ **keywords:** DevOps, Automation, CI/CD, Docker, Beer]
]

]

.center.pic-circle[
<img src="img/tcarreira.png" width="180">

<img alt="Music" src="img/piano.svg" width="80" margin="20">
    

<img alt="Puzzles" src="img/runer-silhouette-running-fast.svg" width="80" margin="20">
]

.center.no-bullets.tiny[
- **telegram:** @tcarreira
- **github:** https://github.com/tcarreira
- **twitter:** @tiagogcarreira
]

]

---

# Agenda

- Docker Best Practices
  - layer caching
  - keep tidy images
  - image tags
  - Docker as documentation
  - non-root
- inspect image layers
- multi-stage builds
- volumes for developers

---

# Docker Best Practices Improving Dockerfiles

]

---

# Dockerfile

- File
- Set of instructions
- Blueprint of an image

???

Very basically...

# Areas of improvements

- Image size
- (Incremental) build time
- Consistency/Repeatability
- Maintainability
- Security

???

- not limited to...

---

# Example project

Basic Java Spring Hello world web app

```no-highlight
drwxr-xr-x 2  9.4M Dec  3 09:44 .git/
-rw-r--r-- 1   656 Dec  4 12:20 Dockerfile
drwxr-xr-x 2  6.1M Dec  4 09:44 docs/
-rw-r--r-- 1  1.7K Dec  3 09:48 pom.xml
-rw-r--r-- 1  1.0K Dec  4 10:12 README.md
drwxr-xr-x 4   44K Dec  3 09:48 src/
drwxr-xr-x 2   17M Dec  4 09:50 target/
```

???

simple:

- project on git repo
- some files:
  - Dockerfile
  - pom.xml
- some dirs:
  - src
  - target

---

# Let’s improve this Dockerfile

```dockerfile
FROM debian
COPY . /app
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk ssh emacs
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

This dockerfile

- simply copy everything for the container
- install jdk and some tools
- run a built jar

Remember:

- an **Image** is a list of layers
- each **docker command** build one layer on top of the previous

---

# Let’s improve this Dockerfile

```dockerfile
FROM debian
COPY . /app
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk ssh <-->emacs</--> <+>vim</+>
CMD ["java", "-jar", "/app/target/app.jar"]
```

---

# Order matters for caching

```dockerfile
FROM debian
<-->COPY . /app</-->
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk ssh vim
<+>COPY . /app</+>
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

- images are a group of layers
- each dockerfile command builds a layer
- that layer is cached
- a layer is rebuilt if
  - a line is changed
  - files checksum differ
  - previous layer is rebuilt

---

# More specific COPY to limit cache bust

```dockerfile
FROM debian
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk ssh vim
<-->COPY . /app</->
<+>COPY target/app.jar /app.jar</->
CMD ["java", "-jar", "<-->/app/target</-->/app.jar"]
```

???

you don't need all src inside the container

---

# Identify cachable units

```dockerfile
FROM debian
*RUN apt-get update
*RUN apt-get -y install openjdk-8-jdk ssh vim
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

- remember when a layer is cached if line does not change?
- apt-get update will not update forever
- what happens when want to force a new package version? (next slide)

---

# Identify cachable units

```dockerfile
FROM debian
*RUN apt-get update
*RUN apt-get -y install openjdk-8-jdk ssh<+>=1:7.4p1-10+deb9u7</+> vim
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

- remember when a layer is cached if line does not change?
- apt-get update will not update forever

---

# apt-get update + install

```dockerfile
FROM debian
<-->RUN apt-get update</-->
<-->RUN apt-get -y install openjdk-8-jdk ssh vim</-->
*RUN apt-get update \
* && apt-get install -y \
* openjdk-8-jdk \
* ssh \
* vim
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

- always keep apt-get update + install together
- protips:
  - one package per line
  - alphabetical order

---

# Don't pack unnecessary dependencies

```dockerfile
FROM debian
RUN apt-get update \
 && apt-get install -y \
 openjdk-8-jdk<--> \</-->
 <-->ssh \</-->
 <-->vim</-->
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

It's bad practice to ssh into a container 
Use `exec` if needed but only in dev

---

# Use --no-install-recommends

```dockerfile
FROM debian
RUN apt-get update \
 && apt-get install -y <+>--no-install-recommends</+> \
 openjdk-8-jdk
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

Intended most of the times

---

# Remove package manager cache

```dockerfile
FROM debian
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
 openjdk-8-jdk <+>\</+>
 <+>&& apt-get clean \</+>
 <+>&& rm -rf /var/lib/apt/lists/*</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

as a layers are cached after all commands are run, the apt-get cache is never stored

But now the dockerfile is a little more complex

---

# Reuse official images when possible

```dockerfile
<-->FROM debian</-->
<-->RUN apt-get update \</-->
<--> && apt-get install -y --no-install-recommends \</-->
<--> openjdk-8-jdk \</-->
<--> && apt-get clean \</-->
<--> && rm -rf /var/lib/apt/lists/*</-->
<+>FROM openjdk</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

(from next slide)

- Reduce time spent on maintenance (frequently updated with fixes)
- Reduce size (shared layers between images)
- Pre-configured for container use
- Built by smart people
- Bonus: scanned for vulnerabilities on Docker Hub

---

# Reuse official images when possible

```dockerfile
FROM openjdk
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

---

# Use more specific tags

```dockerfile
FROM openjdk<-->:latest</-->
FROM openjdk<+>:8</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

Image documentation: https://hub.docker.com/_/openjdk

???

- when not specified, latest is used
- latest is an anti-pattern:
  - just a tag
  - not guaranteed to exist
  - major version upgrades
  - not always the last published image

---

# Use more specific tags

```dockerfile
FROM openjdk:8
FROM openjdk:8<+>-jre</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

Image documentation: https://hub.docker.com/_/openjdk

???

- we don't need a JDK.
- the java runtime environment is enough

---

# Look for minimal flavors

```dockerfile
FROM openjdk:8-jre
FROM openjdk:8-jre<+>-slim</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

Image documentation: https://hub.docker.com/_/openjdk

???

Many times, there are slimmer versions of the image

---

# Look for minimal flavors

```dockerfile
FROM openjdk:8-jre-<-->slim</-->
FROM openjdk:8-jre-<+>alpine</+>
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

Image documentation: https://hub.docker.com/_/openjdk

???

or even consider using alpine version

- based on alpine
 - linux minimal distro (<5MB)

**disclaimer** there are specific cases wher it cannot be used. Check accordingly to the project

---

# Size does matter

```no-highlight
REPOSITORY  TAG           SIZE
openjdk     8             624MB
openjdk     8-jre         443MB
openjdk     8-jre-slim    204MB
openjdk     8-jre-alpine  83MB
```

- Faster to download/build
- Faster to deploy/upgrade
- Less bandwidth
- Less storage

---

# Where is the elephant?

Tips on how to find larger layers

???

Skip to [Checkpoint 1](#checkpoint-1)

- So, how can I find which layers are responsible for the increased size?

---

```no-highlight
$ docker image history maven-hello-world:initial
IMAGE CREATED BY SIZE
7e79f96f032a /bin/sh -c #(nop) CMD ["java" "-jar" "/ap… 0B
*b85fc4b6eb47 /bin/sh -c apt-get -y install openjdk-8-jd… 668MB
ba3d455f8208 /bin/sh -c apt-get update 16.3MB
ab1a2587e9fa /bin/sh -c #(nop) COPY dir:8db399a5922bd10… 2.75MB
3bbb526d2608 /bin/sh -c #(nop) CMD ["bash"] 0B
*<missing> /bin/sh -c #(nop) ADD file:370028dca6e8ca9… 101MB
```

???

- **docker image history** shows the layers which compose the image
- To be read bottom-up (the last layer in on top)
- The base image is around 100MB
- But the apt-get install part is +650MB

---

```no-highlight
$ docker image history maven-hello-world:checkpoint1
IMAGE CREATED BY SIZE
141498904e4e /bin/sh -c #(nop) CMD ["java" "-jar" "/ap… 0B
*125c66d5e6d4 /bin/sh -c #(nop) COPY file:49d9555ff44c04… 2.7MB
*d4557f2c5b71 /bin/sh -c set -x && apk add --no-cache … 78.6MB
<missing> /bin/sh -c #(nop) ENV JAVA_ALPINE_VERSION… 0B
<missing> /bin/sh -c #(nop) ENV JAVA_VERSION=8u181 0B
<missing> /bin/sh -c #(nop) ENV PATH=/usr/local/sbi… 0B
<missing> /bin/sh -c #(nop) ENV JAVA_HOME=/usr/lib/… 0B
<missing> /bin/sh -c { echo '#!/bin/sh'; echo 's… 87B
<missing> /bin/sh -c #(nop) ENV LANG=C.UTF-8 0B
<missing> /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
*<missing> /bin/sh -c #(nop) ADD file:2ff00caea4e83df… 4.41MB
```

???

as for the last dockerfile,

- the base image is 4.41MB
- apk add is <80MB

---

```no-highlight
*$ docker build --no-cache <+>--rm=false</+> .
Sending build context to Docker daemon 2.841MB
Step 1/4 : FROM openjdk:8-jre-alpine
...
Step 3/4 : RUN chmod +x /app.jar
* ---> Running in 02f7f352548f
...
Successfully built bc434fe0055b
Successfully tagged maven-hello-world:checkpoint1
```

???

- Adicionally, **docker container diff** shows exactly what has changed inside of a container (from the previous image)
- shows systemfs changes

```no-highlight
$ docker container diff d2eef182c025
*C /app.jar
```

???

*for notes only*:

- Take note of the first character
  - A: Add (more bytes)
  - C: Change (wasted bytes)
  - D: Delete (saves nothing, only a filesystem marker in the next layer)

---

# Checkpoint

???

So, returning to our dockerfile

---

# Differences in size

```no-highlight
$ docker images \
  --format "table {{.Repository}}:{{.Tag}}\\t{{.ID}}\\t{{.Size}}"\
  --filter 'reference=maven-hello-world'

REPOSITORY:TAG                  IMAGE ID            SIZE
maven-hello-world:initial       7e79f96f032a        788MB
maven-hello-world:checkpoint    bc434fe0055b        88.4MB
```

???

- Our changes spared around 700MB
- This means:
  - Less storage impact
  - Less network bandwidth
  - Less time spent

---

# Initial dockerfile

```dockerfile
FROM debian
COPY . /app
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk ssh emacs
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

We started with this...

# After some tweaks

```dockerfile
FROM openjdk:8-jre-alpine
COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

... and we are now with this.

- Only simple steps until now
  - specific base image
  - copy only what we need
  - run as before

---

# Sh*t is about to get real

---

# Dockerfile as Documentation

]

---

# Look for reproducibility

```dockerfile
FROM openjdk:8-jre-alpine
*COPY target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

- Let's look for reproducibility
- Where did that `target/app.jar` came from?
  - Looks like it appears by dark magic spells
  - Which only the main sourcerer knows how to build

---

# Build from source

```no-highlight
<-->FROM openjdk:8-jre-alpine</-->
<+>FROM maven:3.6-jdk-8-alpine</+>
<-->COPY target/app.jar /app.jar</+>
<+>COPY pom.xml /app/</+>
<+>COPY src /app/src</+>
<+>RUN cd /app && mvn -e -B package</+>
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

- let's rebuild everything
- aim for dockerfile as documentation

---

# Build from source

```dockerfile
*FROM maven:3.6-jdk-8-alpine
COPY pom.xml /app/
COPY src /app/src
RUN cd /app && <=>mvn -e -B package</=>
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

- Starting with maven as base image
- Running the build command as part of the image build
- Now our dockerfile documents the build process
- But it still needs some optimizations

---

# Don't repeat yourself

```dockerfile
FROM maven:3.6-jdk-8-alpine
<+>WORKDIR /app</+>
COPY pom.xml <-->/app/</-->.
COPY src <-->/app/</--><+>.</+>/src
RUN <-->cd /app && </-->mvn -e -B package
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

- First, we don't need to repeat all those `/app`
- Instead of `cd` into a dir, we use `WORKDIR`

---

# Cache dependencies

```dockerfile
FROM maven:3.6-jdk-8-alpine
WORKDIR /app
COPY pom.xml .
<+>RUN mvn -e -B dependency:resolve</+>
COPY src ./src
RUN mvn -e -B package
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

- we can leverage cache, as seen before
- dependencies change less than code itself
- keep cached dependencies
- helps building time (if we only change the code itself)

---

# Identify build dependencies

- Remove unecessary dependencies from final image

```dockerfile
FROM <=>maven:3.6</=>-jdk-8-alpine
WORKDIR /app
*COPY pom.xml .
*RUN mvn -e -B dependency:resolve
*COPY src ./src
RUN mvn -e -B package
CMD ["java", "-jar", "/app/target/app.jar"]
```

???

After this, we run into the same problem again:

- Do not ship build dependencies

**How can we do this?**

---

# Enter multi-stage builds

---

# Multi-stage builds

```dockerfile
*FROM maven:3.6-jdk-8-alpine <+>AS builder</+>
WORKDIR /app
COPY pom.xml .
RUN mvn -e -B dependency:resolve
COPY src ./src
RUN mvn -e -B package
<-->CMD ["java", "-jar", "/app/target/app.jar"]</-->

*<+>FROM openjdk:8-jre-alpine</+>
<+>COPY --from=builder /app/target/app.jar /app.jar</+>
<+>CMD ["java", "-jar", "/app.jar"]</+>
```

???

- stages are defined by the keyword `FROM`
- we have now 2 stages
- keyword `AS` to name the stage and reference it later

---

# Multi-stage builds

```dockerfile
FROM maven:3.6-jdk-8-alpine <=>AS builder</=>
WORKDIR /app
COPY pom.xml .
RUN mvn -e -B dependency:resolve
COPY src ./src
RUN mvn -e -B package

*FROM openjdk:8-jre-alpine
COPY <=>--from=builder</=> /app/target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

(behind the scenes)

- on `docker build`:
  - get last stage by default (last FROM)
  - it references another stage (as `builder`), so build it

---

# Multi-stage builds

Checkpoint 1

```dockerfile
FROM openjdk:8-jre-alpine
COPY /app/target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

Checkpoint with multi-stage build

```dockerfile
...
FROM openjdk:8-jre-alpine
COPY <+>--from=builder</+> /app/target/app.jar /app.jar
CMD ["java", "-jar", "/app.jar"]
```

???

Last stage is very similar to what we had, but now Dockerfile "documents" the build process

---

# Multi-stage builds uses

- Separate build from runtime environment (lower image size)
- Slight variations on images
- DRY (Don’t Repeat Yourself)
- Build/dev/test/lint environments
- Concurrent stages
- Platform-specific stages

???

- Separate build from runtime environment
  - as the example

more on this is ... 
... out of the scope for today

---

# What about security?

???

- The java is running as root inside the container
- Despite that, there is already an isolation layer when running docker container
- Just for the best practice thing

---

# Running as non-root

]

---

# Add new user

## Debian

```dockerfile
FROM debian:9-slim
RUN mkdir -p /app \
*    && useradd -U \
        -c 'My User' \
        -d /app \
        -s /bin/bash \
        -u 1234 \
        myuser
*USER myuser
```

].right-column[

## Alpine

```dockerfile
FROM alpine:3.8
RUN mkdir -p /app \
*    && adduser -D \
        -g 'My User' \
        -h /app \
        -s /bin/ash \
        -u 1234 \
        myuser
*USER myuser
```

]

???

- You need to create a new user
  - debian: useradd
  - alpine: adduser
- If an app dir exists, assign it to user's Home
- USER myuser

---

# Change files ownership

```dockerfile
FROM openjdk:8-jre-alpine
<+>RUN adduser -D myuser</+>
COPY --from=builder /app/target/app.jar /app.jar
<+>RUN chown myuser: /app.jar</+>
<+>USER myuser</+>
CMD ["java", "-jar", "/app.jar"]
```

???

- note: this `RUN chown` is only academic (no need for it)

But...

???

Everything is running fine.

But...

---

# copy-on-write side effects

```no-highlight
$ docker image history maven-hello-world:checkpoint3
IMAGE         CREATED BY                                   SIZE
4ef7544b7c92  /bin/sh -c #(nop)  CMD ["java" "-jar" "/a…   0B
*5a260de4f985  /bin/sh -c chown myuser: /app.jar            2.7MB
6cea4c3e53bd  /bin/sh -c #(nop) COPY file:a6075c989ff94…   2.7MB
e8c32bbd0bcf  /bin/sh -c adduser -D myuser                 4.81kB
d4557f2c5b71  /bin/sh -c set -x  && apk add --no-cache …   78.6MB
...
```

???

- Where did those 2.7MB come from?
- COW duplicates a file on metadata changes

---

# COPY --chown

```dockerfile
FROM openjdk:8-jre-alpine
RUN adduser -D myuser
COPY --from=builder <+>--chown=myuser</+> /app/target/app.jar /app.jar
<-->RUN chown myuser: /app.jar</-->
USER myuser
CMD ["java", "-jar", "/app.jar"]
```

???

use `--chown` instead

---

```no-highlight
$ docker images
REPOSITORY:TAG                  IMAGE ID            SIZE
maven-hello-world:checkpoint3   4ef7544b7c92        88.4MB
maven-hello-world:checkpoint4   90202191e597        85.7MB
```

???

This case spares ~2MB

```no-highlight
$ docker run -it maven-hello-world:checkpoint4 /bin/sh
/ $ whoami
<=>myuser</=>
/ $ ls -FAlh /
-rwxr-xr-x 1 root root 0 .dockerenv*
-rw-r--r-- 1 <=>myuser</=> <=>myuser</=> 2.6M app.jar
drwxr-xr-x 2 root root 4.0K bin/
...
```

???

With this changes, we have a non-root user

Another quick win :)

---

# Volumes for Developers

???

- Devs users need to bind mount
- But permissions...

---

# Bind mount your code

```conf
version: '3.3'
services:
    app:
        image: 'myapp:latest'
        build:
            context: .
            dockerfile: Dockerfile
        volumes:
            - ./app:/app
        ports:
            - '8080:80'
            - '4000:4000' # debug
```

???

- Usually for Development we can bind mount part of our workdir
- But...

---

# But... ooops

```no-highlight
Error accessing /app: permission denied
```

???

- But you could get some error like this.
- Or you could end up with some files with a different owner on your workdir

- The container user has a UID/GID
- May differ from your UID/GID
- Best scenario: your user UID/GID matches container user UID/GID

---

# Fixing UID/GID

Possible solutions:

- Run everything as root
- Change permissions to 777
- Adjust each developers uid/gid to match image
- Adjust image uid/gid to match developers
- Change the container uid/gid from `run` or `compose`

???

- Lots of bad solutions that will result in:
  - Writing files as root or another user on your laptop
  - Making a different image per developer
  - Or still missing some edge cases (existing files owned by user in image)

---

# Fixing UID/GID

Possible solutions:

- Run everything as root
- Change permissions to 777
- Adjust each developers uid/gid to match image
- Adjust image uid/gid to match developers
- Change the container uid/gid from `run` or `compose`
- **"... or we could use a shell script"** - Some Ops Guy

???

- There is a workaround with a shell script

---

# Disclaimer

The following slide may not be suitable for all audiences

???
- Those developers that are disturbed by shell scripts may want to turn away for this next slide

---

```no-highlight
# update the uid
if [ -n "$opt_u" ]; then
* OLD_UID=$(getent passwd "${opt_u}" | cut -f3 -d:)
* NEW_UID=$(ls -nd "$1" | awk '{print $3}')
  if [ "$OLD_UID" != "$NEW_UID" ]; then
    echo "Changing UID of $opt_u from $OLD_UID to $NEW_UID"
*   usermod -u "$NEW_UID" -o "$opt_u"
    if [ -n "$opt_r" ]; then
*     find / -xdev -user "$OLD_UID" -exec chown -h "$opt_u" {} \;
    fi
  fi
fi
```

This is part of `fix-perms.sh`

???
- This is part of a `fix-perms` shell script I package into my base image
- The first highlighted line gets the UID of the user inside the container
- The second highlight gets the UID of the file or directory mounted as a volume
- If those two UID's do not match, I change the container to match the host
  with the `usermod`
- And after running that `usermod`, I run a `chown` on any files still owned
  by the old UID inside the container

---

```no-highlight
#!/bin/sh
if [ "$(id -u)" = "0" ]; then
  fix-perms -r -u app -g app /code
  exec gosu app "$@"
else
  exec "$@"
fi
```

This is part of `/entrypoint.sh`

???

- That entrypoint checks if I'm root, and if so, fixes the /code permissions to
  match the app container uid
- Then I have this `exec gosu` that drops from `root` to the `app` user and runs
  the cmd
- In prod where I don't run as root, and have matched the prod uid's to match
  the image, this gets skipped and I exec the command
- The end result is the cmd is running as the user as pid 1, all evidence of
  the entrypoint is gone from the process list, making it transparent

---

```conf
version: '3.3'
services:
    app:
        image: 'myapp:latest'
        build:
            context: .
            dockerfile: Dockerfile
        volumes:
            - ./app:/app
        ports:
            - '8080:80'
            - '4000:4000' # debug
*       user: "0:0" # dev only
```

???

- The developer compose file is the same as before with one addition, the user is set to root.
- The production compose file wouldn't have any of this
  - Prod will run as default app, with no volume mounts, build, or cmd.
- Note when you restart the container, the app user has already been modified
  from before, so all the uid/gid changing steps get skipped on the second pass

---

- Developers can run the same image and compose file on multiple systems
- App runs with the developers individual uid/gid
- Changes to `./app` are owned by the developer
- Same image in prod can run without ever needing root

???

- The prod compose file wouldn't change the user, cmd, or volumes

---

# How about debug with Docker?

---

# Debug with Docker

- Just like remote debug
  - Aware of the things under the hood
  - Expose debugging port
  - Configure debugger port

???

- It's less simple than straight forward development
- It's just like remote debugging
- You need to know how it works
- You need to know ports
- **Example Demo?**

---

# More tips?

---

---
# Prune

```no-highlight
*$ docker system prune
WARNING! This will remove:
        - all stopped containers
        - all networks not used by at least one container
        - all dangling images
        - all build cache
```
???

- Be careful running this in Prod
- Consider labeling your containers
- Some run this and complain that their drives are still full

--
What this doesn't clean by default:
- Running containers (and their logs)
- Tagged images
- Volumes

---

# Prune

```no-highlight
*$ docker system prune <+>-f --all --volumes</+>
WARNING! This will remove:
 - all stopped containers
 - all networks not used by at least one container
 - all dangling images
 - all build cache
```
```no-highlight
Options:
 -a, --all Remove all unused images not just dangling ones
 -f, --force Do not prompt for confirmation
 --volumes Prune volumes
```
---

# Prune - YOLO

```no-highlight
$ docker run -d --restart=unless-stopped --name cleanup \
    -v /var/run/docker.sock:/var/run/docker.sock \
    docker /bin/sh -c \
    "while true; do docker system prune -f; sleep 1h; done"
```

???

- If you're going to ignore all my words of caution, here's how you can
  automate the accidental deletion of data.
- Tip from Bret Fisher
- I call this YOLO for a reason
- Be careful since this removes stopped containers and untagged images
  - Untagged images includes your build cache

---
template: impact
name: logs

???

- One thing that prune doesn't clean are container logs
- If you have long running containers, they can fill your disk

---

# Clean Your Logs

```no-highlight
$ cat docker-compose.yml
version: '3.7'
services:
  app:
    image: sudobmitch/loggen
    command: [ "150", "180" ]
*   logging:
*     options:
*       max-size: "10m"
*       max-file: "3"
```

???

- In case you don't run containers by hand, you can set these flags in a
  compose file
- That's a lot of typing to do per service the compose file. What if we had
  a dozen services? -> https://sudo-bmitch.github.io/presentations/dc2019/tips-and-tricks-of-the-captains.html#p11

---

# Clean Your Logs
- Best option to prevent container logs from filling disk space
```no-highlight
$ cat /etc/docker/daemon.json
{
  "log-opts": {"max-size": "10m", "max-file": "3"}
}
$ systemctl reload docker
```
???
- Does not effect already running containers
- Can be overridden per container
- Docker engine does need to be reloaded to take effect

---

# Network Debugging

- Debugging networks from the host doesn't see inside the container namespace
- Debugging inside the container means installing tools inside that container

???

- So we can now run our containers as home, work, and the coffee shop
- But next we want to debug the network, and none of our network debugging
  tools understand the namespaced networking. If you check for open ports on
  the host, that doesn't help us debug what's happening inside the container's
  network namespace.

--
- Networks in docker come in a few flavors: bridge, overlay, host, none
- You can also configure the network namespace to be another container

???

- The trick to debugging in a network namespace comes down to the types of
  docker networks, you probably know bridge, overlay, and host
- The "container" network type attaches one container to another's namespace
- K8s people know this as pod networking

---

# Network Debugging

```no-highlight
$ docker run --name web -p 9999:80 -d nginx

*$ docker run -it --rm --net container:web \
    nicolaka/netshoot ss -lnt
State     Recv-Q Send-Q Local  Address:Port    Peer Address:Port
LISTEN    0      128           *:80            *:*
```

???

- Nothing was ever installed in nginx, but we were able to use all of our
  network debugging tools as if we were in the same network
- We can also use this to test connections between containers over docker
  networking, e.g. ping, curl, nslookup, etc, as one container talking to
  another, to know if the issue is our application or our network configuration

---

# DOCKER_BUILDKIT=1

---

# BuildKit Features For Everyone

- GA in Docker 18.09
- Context only pulls needed files that have changed from previous builds
  - And it only pulls files you ADD or COPY, not the entire context folder
- Multi-stage builds use a dependency graph
- Cache from a remote registry
- Cache pruning has options for age and size to keep

???

- Context is effectively an rsync
- Dependency graph means buildkit only builds stages needed to get to the
  target. If you have a multi-stage build with a test stage in the middle,
  buildkit will likely skip right over that stage.
  - You can always explicitly build any target
- Caching from a registry is useful for temporary build environments (cloud)

---

# BuildKit Experimental Features

- Change your frontend to any parser you want, implemented with a Docker image
- Bind Mounts, from build context or another image
- Cache Mounts, similar to a named volume
- Tmpfs Mounts
- Build Secrets, file never written to image filesystem
- SSH Agent, private Git repos

???

- You can build your own Dockerfile parser, it's just an image
- The parser itself is a `# syntax=` line at the top of the Dockerfile
  - "Parser directive" in Dockerfile notation
  - Change the parser, per image, add new features to old docker engine
- Other bullets are a `RUN --mount` command,
  mounted directories do not get included in the image.
- Bind: to context or image, microscanner, large data processing
- Cache: Maven's m2, Golang module and git cache, apt package
  download, npm, all saved from previous builds
- Secrets: ssh key, aws credentials, injected as a file that doesn't get
  written to image
- SSH: if your key is password protected, use ssh-agent

---

```no-highlight
*# syntax=docker/dockerfile:experimental

FROM golang:1.11-alpine as build
RUN  apk add --no-cache git ca-certificates tzdata
RUN  adduser -D appuser
WORKDIR /src
COPY . /src/
*RUN  --mount=type=cache,id=gomod,target=/go/pkg/mod/cache \
*    --mount=type=cache,id=goroot,target=/root/.cache/go-build \
     CGO_ENABLED=0 go build -o app .
USER appuser
CMD ./app
```

???

- Note the first line, that is not a comment, it's a parser directive that is
  used by buildkit to change the frontend parser
- The RUN command has two cache mounts, these are the same two directories we
  saw in the diff output before
- Once you start using experimental features, you won't be building this image
  without BuildKit, those `--mount` args are not supported by the classic build

---

# Enable BuildKit

```no-highlight
$ export DOCKER_BUILDKIT=1
$ docker build -t your_image .
```
???
- To run BuildKit, you just export an environment variable and build like normal

```no-highlight
$ cat /etc/docker/daemon.json
{ "features": {"buildkit": true} }
```

???
- Or to make BuildKit the new default, you can configure the daemon.json with
  the above "features" setting
- Support for tools like docker-compose is being worked on.
  - Build with `docker build` in CI or a script anyway.
- Even without experimental features like `--mount`, the backwards compatible
  changes are worth the upgrade: pulling only the parts of the context that
  changed and are needed, dependency graph for multi-stage builds, remote
  registry caching, and improved cache pruning

---

# What next

]

---

# DevOps Best practices

- Documentation/maintainability
  - Dockerfile
  - docker-compose
  - Use tags and versioning
- Production ready
  - Keep it short and fast
  - Keep it safe (no root)
  - Use secrets
  - Do not rely on `exec` into containers

]
.right-column[

- Development ready
  - Docker volumes
  - Facilitate debug in dev (only)
- Multi-environment ready
  - ENV variables
  - Smart defaults
- High ~~uptime~~ availability by design
  - Healthchecks
  - Concurrency ready
  - Monitoring support

]

???

- Document with Dockerfile/docker-compose
- Use smart tags and versioning (make it obvious what is running)
- For DEV, allow debug (expose debug ports) and use volumes (quick change what is inside)
- Configuration over environment variables
- DO NOT thing about uptime, but THINK about avalability:
  - healthchecks
  - allow concurrency
  - good monitoring: send logs and expose metrics

---

# References

[1] Dockerfile Best Practices - Sebastiaan van Stijn 
https://www.slideshare.net/Docker/dceu-18-dockerfile-best-practices

[2] Tips and Tricks From A Docker Captain - Brandon Mitchell 
https://github.com/sudo-bmitch/presentations

[3] Docker Presentations 
https://www.slideshare.net/Docker/presentations

---

# That's all folks
<img src="img/warner-bros-michigan-j-frog-logo.png" height="340px">

---

# Reach me

<img src="img/presentation-url-qr2.png" height="250px"> 
.tiny[
<a href="https://tcarreira.github.io/presentations/ubucon-europe-2019/docker2.html">
https://tcarreira.github.io/presentations/ ubucon-europe-2019/docker2.html
</a>
]
]

]

.rigth-column[
 
.center.no-bullets.tiny[
- **telegram:** @tcarreira
- **github:** https://github.com/tcarreira
- **twitter:** @tiagogcarreira
]

]

???

Reach me on telegram

---

# Docker - Tips and Best Practices