Return to blog

How to optimize Docker image with layer caching

hero image for How to optimize Docker image with layer caching

Optimizing things is important part of development process. Even when we are living in the world of fast Internet connections, we still don't want to spend few seconds watching white screen and waiting for website content. And it is just a few seconds! Not that far ago we had to wait much more time, but now everything must be accessible immediately. Let's take a look on Docker image building optimization this time.

If anyone here do not know what Docker is, I will explain it in other article... or you can simply go to official site.

In Docker there is something called 'layer caching'. Each line in Dockerfile generates its own layer during building process. The layers are cached and reused between different building processes if no changes are detected. Let's take a look on this simple Dockerfile I just created for very simple ExpressJS project created from this tutorial:

FROM node 
WORKDIR /usr/src/app 

COPY index.js . 
COPY package.json . 

RUN npm install 
CMD node index.js

This Dockerfile takes node image and make some simple changes. Copies index.js, packages.json and than runs npm install to download all necessary packages. CMD will run this application on container start.

As we agreed before, every line creates own cached layer. Let's try to build this Docker image and see what is happening.

Step 1/6 : FROM node
latest: Pulling from library/node
3d77ce4481b1: Pull complete
534514c83d69: Pull complete
d562b1c3ac3f: Pull complete
4b85e68dc01d: Pull complete
f6a66c5de9db: Pull complete
7a4e7d9a081d: Pull complete
f667553445de: Pull complete
043b4ddce372: Pull complete
Digest: sha256:e54383277abcbe3142f8e487c44fe160a8297131ac1767aba7eafa42432900d8
Status: Downloaded newer image for node:latest
---> f697cb5f31f8
Step 2/6 : WORKDIR /usr/src/app
Removing intermediate container 8b7212515972
---> 39a81767a764
Step 3/6 : COPY index.js .
---> 88a1f28c2a10
Step 4/6 : COPY package.json .
---> 589d8333f6f3
Step 5/6 : RUN npm install
---> Running in 53a63f3773e4
added 50 packages in 2.279s
Removing intermediate container 53a63f3773e4
---> c4fc466e4d0d
Step 6/6 : CMD node index.js
---> Running in 0a0385485dcb
Removing intermediate container 0a0385485dcb
---> 9f9d61cdb69b
Successfully built 9f9d61cdb69b
Successfully tagged codetain/docker-cache:latest

Docker image from this Dockerfile was built completely from scratch now, so it took 1 minute to build. Of course, when we have node image on disk it takes about 4 seconds to build rest of the image. Let's try to build again now.

Step 1/6 : FROM node
---> f697cb5f31f8
Step 2/6 : WORKDIR /usr/src/app
---> Using cache
---> 39a81767a764
Step 3/6 : COPY index.js .
---> Using cache
---> 88a1f28c2a10
Step 4/6 : COPY package.json .
---> Using cache
---> 589d8333f6f3
Step 5/6 : RUN npm install
---> Using cache
---> c4fc466e4d0d
Step 6/6 : CMD node index.js
---> Using cache
---> 9f9d61cdb69b
Successfully built 9f9d61cdb69b
Successfully tagged codetain/docker-cache:latest

A lot less line of scripts, huh? As you can see, there is 'Using cache' text in multiple lines and whole process took less than 1s. This is the power of layer caching. Nothing here was built from scratch, every layer comes from cache.

But let's make some changes in index.js and build image again.

Step 1/6 : FROM node
---> f697cb5f31f8
Step 2/6 : WORKDIR /usr/src/app
---> Using cache
---> 9689b4141c8d
Step 3/6 : COPY index.js .
---> 2845e6728ddd
Step 4/6 : COPY package.json .
---> 6e12f71001e9
Step 5/6 : RUN npm install
---> Running in 2c12736b4503
added 50 packages in 2.22s
Removing intermediate container 2c12736b4503
---> 2266e1c4bb64
Step 6/6 : CMD node index.js
---> Running in fcd321e0274f
Removing intermediate container fcd321e0274f
---> 8beca55dd3ed
Successfully built 8beca55dd3ed
Successfully tagged codetain/docker-cache:latest

Oh oh... what just happened? Starting from 'Step 3/6: COPY index.js .' line - everyting was built without cache. This is expected Docker behaviour! If any layer must be created from scratch because of some change in source files - every next layer is built from scratch too. Can we optimize this process? Sure! And it is very simple - put less frequently changing steps on top of Dockerfile (let them be executed as soon as possible), and frequently changing steps on bottom (end of building process). In this particular case, index.js file (or any file with code) will change probably more often than package.json, so let's make a simple change in Dockerfile:

FROM node

WORKDIR /usr/src/app

COPY package.json .
RUN npm install

COPY index.js .

CMD node index.js

Now let's build our image, than make some change in index.js, and build image again. That's a result:

Step 1/6 : FROM node
---> f697cb5f31f8
Step 2/6 : WORKDIR /usr/src/app
---> Using cache
---> 9689b4141c8d
Step 3/6 : COPY package.json .
---> Using cache
---> b96c69b7cf5f
Step 4/6 : RUN npm install
---> Using cache
---> 275c0e0330a5
Step 5/6 : COPY index.js .
---> d09438362d8f
Step 6/6 : CMD node index.js
---> Running in 2e5427b59e8c
Removing intermediate container 2e5427b59e8c
---> 7ee58be2decd
Successfully built 7ee58be2decd
Successfully tagged codetain/docker-cache:latest

Et voila! Only 5th and 6th step is built from scratch - and it takes a lot less time than running npm install command, etc. I know, I know - results are not that spectacular, as you probably expected, because we are saving only few seconds here, but imagine that you have a lot bigger project with hundreds or thousands of files. Optimizing processes like building Docker images in your deployment or development workflow saves you a lot of time, resources and - obviously - money, so keep it in mind next time when you will be creating new Dockerfiles in your project.

GitHub repo: https://github.com/codetain/docker-caching

As a reliable software company we’re focused on delivering the best quality IT services. However, we’ve discovered that programming skills give us a very particular opportunity...

.eco profile for codetain.eco

Reach Us

65-392 Zielona Góra, Poland

Botaniczna 70

© 2015-2024 Codetain. All rights reserved.

cnlogo