WIP: One-container Recipes
This is a Work In Progress (WIP) post. I explain what I’m working on that is related to automating software development, or related to software development in general.
First, that performance issue …
In the last post I noted that I saw the Terminal process on my Mac jump up to 100% CPU during a
docker build while the
sinatra gem was being installed. After finding the time for a closer investigation, I find that it is not Terminal that was using all that CPU but rather the VBoxHeadless process (i.e., the guest VM process).
I use an application called iStat Menus to provide system and device status information in the OS X menu bar. The CPU utilization menu widget will list the top 5 processes consuming CPU and it was report Terminal at 100%. But that was somehow inaccurate. When I use the OS X Activity Monitor (something like Windows Task Manager) I see the real metrics, which does show the guest VM consuming all that CPU, not my Terminal session.
So, there is no performance problem. At least not the one I had identified.
My Docker recipe repository
I’m keeping a record of what I do as I’m learning to use containers. It’s in a GitHub repository
JeNeSuisPasDave/docker-recipes. Currently the
t-dev/llvmrecipe branch is where active development is going on.
That branch name reflects my initial idea to put the LLVM tool chain on a container and use that container for experimenting with LLVM and clang. It quickly became evident that was too ambitious and I needed to start smaller, and so I did, trying many simple scenarios.
I’m not going to go into every recipe. But the example from the last post is there, in
recipes/Docker Demo - simple web app. And the second example from the Lees-Miller gentle introduction to containers and Docker is also there, in
recipes/Docker Demo - latex runner.
Need for local package sources
While going through these examples and trying other experiments, I noticed a couple things:
The network and remote packages sources are getting a workout.
Creating well-formed containers typically requires attempting many variations of the Docker file. Running
docker builda lot will often cause the same packages to be repeatedly downloaded from remote package sources.
There’s some awkwardness when working with Docker on Windows. For example:
- The Docker container is Linux and expects Unix line endings (lf). But on Windows the local files that might get COPY-ed into the image may have Windows line endings (crlf), and that can cause breakage within the container.
- Local files COPY-ed into the image will have “
-rwxr-xr-x” permissions in the container.
I also ran into a problem where some Ruby gems could not be fetched from a remote package source. This was almost certainly a problem with the network at my office (where I’m using Windows 10), but does cause me to be concerned about relying on remote package sources.
That issue and issue #1 above led me to decide to run my own local package sources for Ruby Gems and Node packages (and probably Python packages). By running local package sources I can avoid over-utilization of the Internet connection. I can also be much more certain of the availability of the package sources. With local package sources I think building and using container images will be much more reliable.
Unix line endings
Issue #2 is a problem only when running the Docker client on a Windows system. If your
Dockerfile COPYs files from your local system into the Docker image, you’ll get a warning like this (on a Windows system):
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have ‘-rwxr-xr-x’ permissions. It is recommended to double check and reset permissions for sensitive files and directories.
I handled this in the
docker-recipes repository by configuring a
.gitattributes file in the project to force all files to use Unix line endings. And I also created a bash script to reset the file permissions to the expected value; that script is RUN after the files are COPYed.
I use Sublime Text 3, so I also configured the ST3 project file to enforce
unix line endings.
In the next post I hope to demonstrate a Ruby gem package source server running in a container.