One of the important challenges with Docker is to get used to the image layers and the layered file system. It quickly happens that you unintentionally have too much data in an intermediate layer. Either log files, installation software or login credentials. Whereby the first two “only” blow up the Docker image unnecessarily, while the last point can be a major security vulnerability. It also happens to me when I build Docker images for Oracle Unified Directory. See my blog post on Oracle Unified Directory on Docker.
Problem
Each instruction in the Dockerfile adds a layer to the image, and you need to remember to clean up any artifacts you don’t need before moving on to the next layer. If you do use COPY or ADD in particular a clean up is not possible. Every credential file or software package which is copied during build will remain. Later attempts to remove the intermediate files will only result in the corresponding files no longer being visible in the next layers.
Although there is a way to work around this by using the new build parameter --squash. Squash does merge newly built layers into a single new layer. But --squash is only available in the latest Docker releases. In older releases not at all or at best as experimental feature. Beside this it also has some other downside eg. losing the history or intermediate layers, issue with ONBUILD command etc. Squash does also not help if you specify your credentials as build arguments via ARG.
So why not make small, secure and clean Docker image at first place.
Idea
Rather than put the software packages or credential files to the Docker build context and using COPY or ADD, we will download them in a RUN command using curl. But from where? Oracle software can not be downloaded unattended without credentials. And we do not want to set up a web server just for software, secrets, credentials, etc.
Hei, you are using Docker. Setting up a local web server is a pice of cake. 🙂
- Put your software, credentials, etc in a dedicated folder
- Run a Docker container with an Apache HTTP server and make sure it has access to the folder mentioned before
- Change your Dockerfile to download the software or credentials for the HTTP server
- Make sure that you
docker buildcan get access to the intermediate HTTP server
Solution
Let’s see how I did use a local HTTP sever to set up small Oracle Unified Directory Docker images.
HTTP Server
Create a folder with all required Oracle software, patch’s etc. But make sure, that you Docker can use this folder as Docker volume.
ls -alh /Data/vm/docker/volumes/orarepo total 5009896 drwxr-xr-x 11 oracle staff 352B 26 Mär 23:52 . drwxr-xr-x 4 oracle staff 128B 26 Mär 22:50 .. -rw-r--r-- 1 oracle staff 1,5G 26 Mär 23:03 p26269885_122130_Generic.zip -rw-r--r-- 1 oracle staff 404M 26 Mär 22:55 p26270957_122130_Generic.zip -rw-r--r-- 1 oracle staff 94M 26 Mär 22:49 p26540481_111230_Generic.zip -rw-r--r-- 1 oracle staff 157M 26 Mär 22:45 p26724938_111170_Linux-x86-64.zip -rw-r--r-- 1 oracle staff 56M 26 Mär 22:55 p27217121_904_Linux-x86-64.zip -rw-r--r-- 1 oracle staff 52M 26 Mär 22:56 p27217289_180162_Linux-x86-64.zip -rw-r--r-- 1 oracle staff 1,3M 26 Mär 22:44 p27438258_122130_Generic.zip -rw-r--r-- 1 oracle staff 56M 26 Mär 22:54 p27478886_100000_Linux-x86-64.zip -rw-r--r-- 1 oracle staff 52M 26 Mär 22:53 p27638647_180162_Linux-x86-64.zip
Get your revered HTTP server. For this case I do use the official Apache HTTP server-based on alpine linux. This image is way smaller and more than enough for our purpose.
docker pull httpd:alpine alpine: Pulling from library/httpd 605ce1bd3f31: Pull complete 6e4ededbced2: Pull complete 03b3c72c9962: Pull complete bf08478b6930: Pull complete 222d70b58166: Pull complete Digest: sha256:80d69271825a27c41f41609707095a1cdec381d22f772511ae6e30156c2b788f Status: Downloaded newer image for httpd:alpine
Start a container for the HTTP server. Define a hostname, volume and external http port. For more information on configuration, see the official httpd Docker image.
docker run -dit --hostname orarepo --name orarepo \ -p 8080:80 \ -v /Data/vm/docker/volumes/orarepo:/usr/local/apache2/htdocs/ \ httpd:alpine
Get the IP address with docker inspect of the orarepo for later use.
orarepo_ip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' orarepo)
A test via curl on command line curl http://localhost:8080 or with your favorite browser will show the files mentioned above. Be aware this test does access the HTTP container from your host network via exposed port 8080.
Dockerfile
Now we just have to adopt the Dockerfile to make sure it does get the software from the HTTP server orarepo. See excerpt from my Dockerfile.
... RUN curl -f http://orarepo/p26270957_122130_Generic.zip \ -o /tmp/download/p26270957_122130_Generic.zip && \ ...
To be more flexible and allow both local files as well download via curl I did extend my RUN command with an extra file check [ -s ... ]. In this case it first check’s if the file is available and not zero. If the file is not available it will use curl to download the file from orarepo. See excerpt from my Dockerfile.
... COPY p26270957_122130_Generic.zip* /tmp/download/ ... RUN [ -s "/tmp/download/p26270957_122130_Generic.zip" ] || \ curl -f http://orarepo/p26270957_122130_Generic.zip \ -o /tmp/download/p26270957_122130_Generic.zip && \ ...
If the OUD software package p26270957_122130_Generic.zip is part of the build context it will be copied to the image and used to build and setup OUD. In case it is not part of the build context the file check will fail and start to use curl.
Build using COPY
Let’s build the Docker image with the software package copied during build. Check the build context.
ls -alh total 858168 drwxr-xr-x 9 oracle staff 288B 28 Mär 09:31 . drwxr-xr-x 6 oracle staff 192B 19 Mär 14:19 .. -rw-r--r-- 1 oracle staff 4,9K 27 Mär 21:40 Dockerfile -rw-r--r-- 1 oracle staff 225B 19 Mär 11:18 install.rsp -rw-r--r-- 1 oracle staff 63B 19 Mär 10:57 oraInst.loc -rw-r--r-- 1 oracle staff 404M 28 Mär 09:31 p26270957_122130_Generic.zip -rw-r--r-- 1 oracle staff 754B 12 Mär 14:18 p26270957_122130_Generic.zip.download drwxr-xr-x 6 oracle staff 192B 20 Mär 11:46 scripts
And run docker build.
docker build -t oracle/oud:12.2.1.3.0-copy .
Build using curl
Now let’s build the docker image using curl and not the local software package. For this p26270957_122130_Generic.zip has to be removed from the build context. Additionally the Docker build requires the IP of the orarepo, which is used to download the software image.
Check the build context.
rm p26270957_122130_Generic.zip ls -alh total 858168 drwxr-xr-x 9 oracle staff 288B 28 Mär 09:31 . drwxr-xr-x 6 oracle staff 192B 19 Mär 14:19 .. -rw-r--r-- 1 oracle staff 4,9K 27 Mär 21:40 Dockerfile -rw-r--r-- 1 oracle staff 225B 19 Mär 11:18 install.rsp -rw-r--r-- 1 oracle staff 63B 19 Mär 10:57 oraInst.loc -rw-r--r-- 1 oracle staff 754B 12 Mär 14:18 p26270957_122130_Generic.zip.download drwxr-xr-x 6 oracle staff 192B 20 Mär 11:46 scripts
And run docker build with --add-host. Add host does use the variable defined above for the IP address of the orarepo.
docker build --add-host=orarepo:${orarepo_ip} -t oracle/oud:12.2.1.3.0-curl .
The Docker images
When we compare the two image we see, that they differ by around 400MB. More or less the size of the OUD software package.
docker images REPOSITORY TAG IMAGE ID CREATED SIZE oracle/oud 12.2.1.3.0-curl f7c80fa69db3 2 minutes ago 754MB oracle/oud 12.2.1.3.0-copy a5e1751d534d 7 minutes ago 1.18GB
Docker history for the image oracle/oud:12.2.1.3.0-copy does also show the size of the COPY layer.
docker history oracle/oud:12.2.1.3.0-copy IMAGE CREATED CREATED BY SIZE COMMENT ... 07614f386e0f 16 minutes ago /bin/sh -c #(nop) COPY multi:b8206d7811ce917… 424MB 832c9d0bf308 34 hours ago /bin/sh -c #(nop) COPY multi:58a01d5459f0ac6… 20kB 9c9531205281 34 hours ago /bin/sh -c groupadd --gid 1000 oracle && … 8.29MB 96278dfe7c12 34 hours ago /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:… ...
Conclusion
With little effort it is possible to create secure and especially small Docker images. Creating a HTTP server to share software or credentials during build is a piece of cake. Reducing the docker image by 400MB is nice. Depending on the Oracle software this will be even more. Although the downside is, that this does not work for automated builds on Docker Hub. but I’m still working on that 🙂
References
Below you find a few references related to the topics discussed in this post:
- GitHub repository to build Oracle Unified Directory Docker images oehrlis/docker
- GitHub repository for the official Oracle Docker images oracle/docker-images
- Official image of the Apache HTTP Server on Docker Hub https
- Docker docs Dockerfile reference
- Docker docs Best practices for writing Dockerfiles
- Docker docs Use multi-stage builds
- Docker docs About storage drivers
- Blog post on docker build squash
- Blog post on docker build with private credentials
Pingback: DOAG 2018 SIG Security – Oracle Unified Directory on Docker | OraDBA
What about mounting the download directory into the image?
Hi
It is not possible to mount volumes during docker build. Beside the download one could also use the squash option during build or use multi-stage builds.
Cheers
Stefan