Docker on Roland Huß

First look at Jib

Wed, 11 Jul 2018 00:00:00 +0000

As soon as Google’s blog post “Introducing Jib — build Java Docker images better” was online, all my channels went crazy about Jib. That was a bit surprising as Jib was started over one year ago but with this blog post this project rockets with more than 1000 new GitHub stars within one day. Crazy.

I got a lot asked yesterday how Jib compares to fabric8io/docker-maven-plugin (d-m-p) or fabric8io/fabric8-maven-plugin which includes d-m-p.

Let me try to shed some light on the differences and pro and cons of both approaches.

How Jib works
#

Let’s first have a quick look what Jib offers today and what makes it unique.

Looking at the Jib Maven plugin it currently supports three goals:

jib:dockerBuild for assembling an image and loading it to a Docker daemon.
jib:build for assembling the image and pushing it to a Docker registry.
jib:exportDockerContext for creating a Dockerfile along with all the files required for performing a build against a Docker daemon.

The unique asset of Jib is that it does all of this without consulting a Docker daemon by creating all image layers and metadata locally, conforming to either the Docker or OCI format. And that all directly with plain Java code.

Also, Jib assumes a very opinionated project setup with a so-called flat classpath app with main class, dependencies and resources all in different artefacts. Compare this to fat jars popularised by Spring Boot, where the application code, resources and dependencies are all stored in the same jar file. There are some drawbacks of flat classpath app, but one benefit is, that you can organise the various files into several layers in your image, putting the one which is changing less (like dependencies) at the bottom of the layer stack. That’s what Jib does: It puts all dependency jars into one layer, all resource files (like property files to be loaded from the classpath) in another and the application classes into a third layer. All of these layers get aggressively cached locally. That way, you can be much faster when recreating images than with fat jars which apparently can be stored only in one single layer.

But let’s have a look how Jib works in detail. The steps performed by a jib:dockerBuild or jib:build are:

Fetch base image’s layers and cache them locally. By default the base image is gcr.io/distroless/java, but this can be configured.
Create three application image layers for
- Dependencies
- Resources
- Classes
Since these layers are cached, if any of them doesn’t change (which is likely for dependencies), then the layer is not recreated.
Create the application image locally. The two previous steps were performed asynchronously; this step continues when both previous steps have been finished. The ENTRYPOINT of this image is fixed set to:
```
 java <jvm-flags> -cp dep_dir/*:resource_dir/:class_dir/ <main-class>
```
where <jvm-flags> can optionally be configured and <main-class> is the mandatory main class to specify. The information leading to the classpath comes from the underlying Maven or Gradle project information. Also, any Java arguments configured to become the CMD of the image, as well the exposed ports (EXPOSE) can be added, too.
Finally the local image layers along with its meta-data is tarred up, and either loaded into a Docker daemon or pushed to a Docker registry.

How does Jib compare to d-m-p ?
#

Jib is impressive for the use case it supports and brings a fresh spin to the way how Java apps can be packaged into Docker images.

The most significant benefits of Jib are:

Fast for incremental builds when you have a flat classpath application, resulting in three different layers for dependencies, resources and your application classes.
No Docker daemon required, which reduces the build requirements and increases security because the image creation happens without root permissions.
Produces reproducible images by wiping out file owner and modifications date. However, not sure whether e.g. generated timestamps in resource files like properties are wiped out, too.
It supports both, Maven and Gradle.

However, there are also some limitations. Some of them might be tackled in the future, but other might not be changed due to the unique way how Jib works:

Can be only used for simple flat classpath Java applications. There is currently no support for fat jars (i.e. Spring Boot fat jars) nor other packaging formats like WAR file.
Simplistic startup of the application with a plain java call instead of using a full features startup script like run-java-sh.
No additional files like configuration files outside the classpath or agents like jmx_exporter agent can be added (but there is a PR pending for agents).
Fixed classpath order, e.g. doesn’t allow for overwritten resources in dependencies as dependencies are always first on the classpath.
Jib uses a custom XML configuration syntax instead of plain Dockerfile syntax (which I often heard as a major critique about d-m-p which also supports a custom XML configuration, but as alternative to Dockerfiles).

d-m-p provides some additional features which are not supported by Jib, like

Running containers for integration testing (that’s very likely the most prominent difference)
Dockerfile support
docker-compose.yml support
Enhanced authentication support OpenShift and Amazon ECR support
Support for watching code change and then automatically triggering a rebuild of images and restart of containers
Support for arbitrary assembly and base images, including Spring Boot fat jar and JavaEE containers.
Healthchecks

And if you jump to fabric8-maven-plugin, which includes d-m-p for its image building business, you have even more high level features like a zero config mode which analyses your pom.xml and selects opinionated defaults like base images and handcrafted startup scripts, depending on the type of tech stack you are using (like Spring Boot, Thorntail, Vert.x, Tomcat, …)

Next steps …
#

This overview is only a quick glance on Jib. In one of the next posts, I plan to show some real-life examples and also measuring the performance gain by using Jib.

Also, there a plans for d-m-p to add support for Jib backend and other daemonless build systems like img, buildah or kaniko. The mid to longterm plan is to enhance the build abstraction within d-m-p and offer, based on the Java project given, different ways to build images.

BTW, if you are interested in what’s going on in Docker image building business these days, you probably might find this KubeCon presentation as useful as I did. Daemonless FTW :)

Psst, d-m-p also likes GitHub ★ ;-)

docker-maven-plugin might be still useful

Thu, 05 Jul 2018 00:00:00 +0000

Yesterday a blog post Using Docker from Maven and Maven from Docker by Kostis Kapelonis was published which gives some insights on the possible relationships between Docker and Maven. The article makes some essential points really, and gives an overview for the two remaining Docker Maven plugins as well as how Codefresh recommends doing Docker multi-stage builds as the alternative. As I’m the maintainer of the fabric8io/docker-maven-plugin, I’d like to comment on this matter.

I already commented on the original blog post (thanks for approving the comment), but I’m happy to repeat my arguments here again.

The article ditches two docker-maven-plugins before it promotes Docker multi-stage builds for some reasons.

To be honest, I think both approaches have their benefits, but let me comment first on two arguments given concerning the fabric8io/docker-maven-plugin.

There have been cases in the past where Docker has broken compatibility even between its client and server, so a Maven plugin that uses the same API will instantly break as well.

This compatibility issue might be right especially if you use a typed approach to access the Docker REST API which is used by various Docker client libraries. As explained in the post, fabric8 d-m-p accesses the Docker daemon directly without any client library and with not marshalling. This is because it accesses only the parts required for the plugin’s feature set, which also means that json responses are handled in a very defensive and untyped way.

And yes, there was one issue in the early days in 2014 with a backwards-incompatible API change from Docker. This issue could be fixed quite quickly because d-m-p hadn’t to wait for a client library to be updated. However, since then there never has been an issue and for the core functionality that d-m-p uses.

I think the relevance of Docker API incompatibilities is exaggerated in this blog post.

Hopefully, the fabric8 plugin also supports plain Dockerfiles. Even there, however, it has some strong opinions. It assumes that the Dockerfile of a project is in src/main/docker and also it uses the assembly syntax for actually deciding what artefact is available during the Docker build step.

That is simply not true. You can just put a Dockerfile on the same level as the pom.xml, refer to your artefacts in the target/ directory (with Maven property substitution), and then declare the plugin without any configuration. See my other blog post for a short description of how it works.

BTW, the reason for the own XML syntax is a historical one. The plugin started in 2014 when Dockerfile was entirely unknown to Java developers. But Maven plugin XML configuration was (and still is) a well-known business. As time passed by and Docker become more and more popular for Java developers, the Dockerfile syntax is well known now these days, too. So, I completely agree, that you should use Dockerfiles if possible, and that’s why the plugin supports Dockerfiles as a first-class citizen since the recent versions. The next step is to add similar support for docker-compose.yml files for running containers. There is already docker compose support included, albeit a bit hidden.

I agree that multi-stage Docker builds are fantastic for generating reproducible builds, as the build tool (Maven) is used in a well-defined version. However, using a locally installed Maven during development has advantages, too. E.g. the local Maven repository avoids downloading artefacts over and over again, resulting in much faster build times and turnaround times. Of course, you can add caching to the mix for multi-stage builds, but then the setup gets more and more involved. Compare this to using a d-m-p for which you don’t even need a local Docker CLI installed, and you can ‘just start’. For CI builds this properly doesn’t matter much though (and that’s what the blog post is all about I guess).

Other advantages of using fabric8’s d-m-p :

Running all your containers (app + deps) locally without the need of support from a CI system. As a side note, the custom compose-like syntax of codefresh’s CI is not so much different to the custom configuration syntax of fabric8’s d-m-p. It’s custom.
Extended authentication support against various registries (Amazon ECR, Google GCR, …)
Automatic rebuilds during development with docker:watch which increase turnaround times tremendously
Download support files (e.g. startup scripts) automatically by just declaring a dependency in the plugin (blog post pending)
…. and even more stuff.

In the end, your mileage may vary, but having an article conclusion without really trying to compare pros and cons of both approaches is far too biased for me.

Update: Kostis replied to my comment, and an interesting discussion is going over there

When a Dockerfile is just good enough

Thu, 12 Apr 2018 00:00:00 +0000

As you might know, one of my Open Source babies is the one and only fabric8io/docker-maven-plugin (d-m-p). If you already use this Maven plugin, you know, that it is super powerful and flexible to configure. This flexibility comes at a price so that the configuration can become quite complicated. Now, if you only want to build Docker images with Maven, I have good news: Since 0.25.1 d-m-p supports a zero XML configuration mode, the so-called Simple Dockerfile Build mode.

The idea of this mode started with a Twitter discussion:

And actually, it’s true: If all that you want is to build a single Docker image from a Dockerfile, then the initial configuration is indeed too complex.

d-m-p already supports plain Dockerfiles for quite some time, and that even for multiple images. However, you still have to reference those Dockerfiles in the XML configuration of the plugin.

Since 0.25.1 you can now use the so-called Simple Dockerfile Build mode (kudos go to Rohan Kumar for the initial implementation). All you have to do is to add d-m-p to your pom.xml and add a Dockerfile. The smallest possible Maven project for creating a Docker image consists of this pom.xml

<project>
 <modelVersion>4.0.0</modelVersion>

 <groupId>fabric8</groupId>
 <artifactId>smallest</artifactId>
 <version>1-SNAPSHOT</version>

 <build>
 <plugins>
 <plugin>
 <groupId>io.fabric8</groupId>
 <artifactId>docker-maven-plugin</artifactId>
 <version>0.26.0</version>
 </plugin>
 </plugins>
 </build>
</project>

and a Dockerfile alongside this pom:

FROM busybox
CMD ["echo", "Hello", "world!"]

This image does not really much. With mvn docker:build you can build it, with docker run fabric8/smallest you can test it. Or use mvn docker:run so that you even don’t have to provide the image name.

A more realistic Dockerfile could look like

FROM openjdk:jre

ARG jar=target/app-1.0.0-SNAPSHOT.jar

ADD $jar /app.jar
CMD java -cp /app.jar HelloWorld

where we define jar as build arg in the Dockerfile but also as property in the pom.xml:

<properties>
 <jar>${project.build.directory}/${project.build.finalName}.jar</jar>
</properties>

You can use Maven properties in the Dockerfile which get automatically replaced by docker:build when creating the image. But you can use that Dockerfile also without Maven with docker build . You can’t use Maven properties with . directly as dots are not allowed in Docker build args, therefore we use an extra property. However, a maven-less usage probably does not make much sense when you don’t also build the artefacts. The full example can be found in the dmp GitHub repo.

If you can forgo Docker build args you can use predefined Maven properties directly:

FROM openjdk:jre
ADD ${project.build.directory}/${project.build.finalName}.jar /app.jar
CMD java -cp /app.jar HelloWorld

The image name is auto-generated, but you can set this name also by yourself by setting the property docker.name (and you can use placeholders within this name)

You can even start the container with mvn docker:run although without any additional configuration (e.g. like port mappings). Also, you can docker:push the image.

You can still configure certain aspects like authentication or bind d-m-p goals to default lifecycle phases. Using this mode is very similar to the functionality offered by spotify/dockerfile-maven.

If you need more horsepower, you can gradually expand on this simple setup. Features which are waiting to be discovered are:

Setup of multiple images for running integration tests
Custom networks and volumes for your tests
Using docker-compose files for running the containers directly from the plugin
Exporting the Docker image as an archive
Watch docker containers and restart them when the code changes
All can be configured via properties, too and since with the latest versions, you can mix it with XML configuration.
….

If you are interested in more to find out, then please have a look at the reference manual.

I’m curious what you think about this new mode. Please use the comments below if you want to leave some feedback. In fact, there are concrete plans for d-m-p to include the generators from the fabric8-maven-plugin functionality of autodetecting the tech stacks used for creating opinionated Docker images.

docker-maven-plugin moves on

Wed, 24 Feb 2016 00:00:00 +0000

rhuss/docker-maven-plugin is dead, long live fabric8io/docker-maven-plugin !

If you follow the Docker Maven Plugin Scene¹, you probably noticed that there has been quite some progress in the last year. Started as a personal research experiment early 2014, rhuss/docker-maven-plugin (d-m-p) has took off a little bit. With more than 300 GitHub stars it’s now the second most popular docker-maven-plugin. With 38 contributors we were able to do 36 releases. It is really fantastic to see that many people contributing to a rather niche product. Many kudos go out to Jae for his many contributions and continued support on fixing and answering issues. Thanks also for always being very patient with my sometimes quite opinionated and picky code reviews :)

However it is now time to ignite the next stage and bring this personal ‘pet’ project to a wider context. And what is better suited here than the fabric8 community ?

Fabric8 is a next generation DevOps and integration platform for Docker based applications, with a focus on Kubernetes and OpenShift as orchestration and build infrastructure. Its a collection of multiple interrelated projects including Maven tooling for interacting with Kubernetes and OpenShift. d-m-p is already included as foundation for creating Docker application images.

I’m very happy that d-m-p has now found its place in this ecosystem where it will continue to flourish even faster.

The fabric8 community is very open and has established multiple communications channels on which you will find d-m-p now, too:

#fabric8 on irc.freenode.net is an IRC channel with a lot of helpful hands (including myself)
A mailing list for more in depth discussions
Issues are still tracked with GitHub issues
d-m-p specific blog posts will go out on the fabric8 blog in the future.

So, what changed ?

rhuss/docker-maven-plugin has been transferred to fabric8io/docker-maven-plugin
The Maven group id has changed from org.jolokia to io.fabric8 for all releases 0.14.0 and later.
CI and release management will be done on the fabric8 platform.

And what will not change ?

d-m-p will always be usable with plain Docker, speaking either to a remote or local Docker daemon. No Kubernetes, no OpenShift required.
I’ll continue to work on d-m-p ;-)

Thanks so much for all the fruitful feedback and pull requests. Keep on rocking ;-)

with more than 15 docker-maven-plugins its probably fair to call it a “scene” ;-) ↩︎

Registry Magic with docker-maven-plugin

Thu, 21 Jan 2016 00:00:00 +0000

Dealing with multiple Docker registries is hard, mostly because the meta information where a image is located is part of a Docker image’s name, which is typically used as an identifier, too.

Let’s see how the rhuss/docker-maven-plugin deals with this peculiarity.

When setting up a Maven build for creating Docker images out of your Java application, the classical way to specify the registry where to push the final Docker image is to bake it into the image’s name. The drawback however is that you couple your build to this particular registry so that it is not possible to push your image to another registry when building the image.

Pull and Push
#

The docker-maven-plugin (d-m-p in short) interacts¹ with Docker registries in two use cases:

Pulling base images from a registry when building images with docker:build or starting images with docker:start
Pushing built images to a registry with docker:push

In both cases you can define your build agnostic from any registry by omitting the registry part in your image names² and specify it externally as meta information. This can be done in various ways:

Adding it to the plugin configuration as an <registry> element. This can be easily put into a Maven profile (either directly in the pom.xml or also in ~/.m2/settings.xml).
Using a system property docker.registry when running Maven
As a final fallback an environment variable DOCKER_REGISTRY can be used, too.

For example,

mvn -Ddocker.registry=myregistry.domain.com:5000 docker:push

When you combine build and push steps in a single call like in

mvn package docker:build docker:push

a pull operation for a base image and a push operation can happen. To allow different registries in this situation the properties docker.pull.registry and docker.push.registry are supported, too, (with the corresponding configuration elements <pullRegistry> and <pushRegistry>, respectively).

When pushing an image this way, the following happens behind the scene (assuming an image named user/myimage and target registry myregistry:5000)

The image user/myimage is tagged temporarily as myregistry:5000/user/myimage in the Docker daemon.
The image myregistry:5000/user/myimage is pushed.
The tag is removed again.

Authentication
#

That’s all fine, but how does d-m-p deal with authentication ? Again, there are several possibilities how authentication can be performed against a registry:

Using a <authConfig> section in the plugin configuration with<username> and <password> elements.
Providing system properties docker.username and docker.password when running Maven
Using a <server> configuration in ~/.m2/settings.xml with possible encrypted password. That’s the most maven-ish way for doing authentication.
Login into the registry with docker login. The plugin will pick up the credentials from ~/.docker/config.json

There are again variants to distinguish between authentication for pulling and pushing images to registries (e.g. docker.push.username and docker.push.password). All the details can be found in the reference manual.

Using the OpenShift Registry
#

OpenShift is an awesome PaaS platform on top of Kubernetes. It comes with an own Docker registry which can be used by d-m-p, too. However, there are some things to watch out for.

First of all, the registry needs to be exposed to the outside so that a Docker daemon outside the OpenShift cluster can talk with the registry:

oc expose service/docker-registry --hostname=docker-registry.mydomain.com

The hostname provided should be resolved by your host to the OpenShift API server’s IP (this happens automatically if you use the fabric8 OpenShift Vagrant image for a one-node developer installation of OpenShift).

Next, it is important to know, that the OpenShift registry use the regular OpenShift SSO authentication, so you have to login into OpenShift before you can push to the registry. The access token obtained from the login is then used as the password for accessing the registry:

# Login to OpenShift. Credentials are stored in ~/.kube/config.json:
oc login

# Use user and access token for authentication:
mvn docker:push -Ddocker.registry=docker-registry.mydomain.com \
 -Ddocker.username=$(oc whoami) \
 -Ddocker.password=$(oc whoami -t)

The last step can be simplified by using -Ddocker.useOpenShiftAuth which does the user and token lookup transparently.

mvn docker:push -Ddocker.registry=docker-registry.mydomain.com \
 -Ddocker.useOpenShiftAuth

The configuration option useOpenShiftAuth again comes in multiple flavours: a default one, and dedicated for push and pull operations (docker.pull.useOpenShiftAuth and docker.push.useOpenShiftAuth).

tl;dr
#

Among all the many docker maven plugins, rhuss/docker-maven-plugin provides the most flexible options for accessing Docker registries and authentication. The gory details can be found in the reference manual which documents registry handling and authentication in detail.

The interaction is always indirectly via the Docker daemon, since a Docker client like d-m-p only talks with the Docker daemon directly. ↩︎
Of course you can the registry part in your image names in which case this registry has always the highest priority. ↩︎

fish-pepper - Docker on Capsaicin

Mon, 07 Sep 2015 00:00:00 +0000

When I had to create multiple Docker base images which only differ slightly for some minor variations I couldn’t avoid to feel quite dirty because of all the copying & pasting of Dockerfile fragments. We all know how this smells, but unfortunately Docker has only an answer for inheritance but not for composition of Docker images. Luckily there is now fish-pepper, a multi-dimensional docker build generator, which steps into the breach.

For example consider a Java base image: Some users might require Java 7, some want Java 8. For running Microservices a JRE might be sufficient. In other use cases you need a full JDK. These four variants are all quite similar with respect to documentation, Dockerfiles and support files like startup scripts. Copy-and-paste might seem to work for the initial setup but there are severe drawbacks considering image evolution or introduction of even more parameters.

With fish-pepper you can use flexible templates which are filled with variations of the base image (like 'version' : ['java7', 'java8'], 'type': ['jdk', 'jre']) and which will create multiple, similar Dockerfile builds.

The main configuration of an image family is images.yml which defines the possible parameters. For the example above it is

fish-pepper:
 params:
 - "version"
 - "type"

The possible values for these parameters are given in a dedicated config section:

config:
 version:
 openjdk7:
 java: "java:7u79"
 fullVersion: "OpenJDK 1.7.0_79"
 openjdk8:
 java: "java:8u45"
 fullVersion: "OpenJDK 1.8.0_45"
 type:
 jre:
 extension: "-jre"
 jdk:
 extension: "-jdk"

Given this configuration, four builds will be generated when calling fish-pepper, one for each combination of version (“openjdk7” and “openjdk8”) and type (“jre” and “jdk”) parameter values.

These value can now be filled into templates which are stored in a templates/ directory. The Dockerfile in this directory can refer to this configuration through a context object fp:

FROM {{ "{{= fp.config.version.java + fp.config.type.extension " }}}}
.....

Templates use DoT.js as template engine, so that the full expressiveness of JavaScript is available. The fish-pepper context object fp holds the configuration and more.

The given configuration will lead to four Docker build directories:

images/
 +---- openjdk7
 | +--- jre -- Dockerfile, ...
 | +--- jdk -- Dockerfile, ...
 |
 +---- opendjk8
 +--- jre -- Dockerfile, ...
 +--- jdk -- Dockerfile, ...

The generated build files can also be used directly to create the images with fish-pepper build. This will reach out to a Docker daemon and create the images java-openjdk7-jre, java-openjdk7-jdk, java-openjdk8-jre and java-openjdk8-jdk.

Alternatively these builds can be used as content for automated Docker Hub builds when checked into Github. The full example can be found on GitHub.

But wait, there is more:

Blocks can be used to reuse Dockerfile snippets and files to include across images. Blocks can be stored locally or referenced via a remote Git repository. Examples for blocks are generic startup scripts or other value add functionality like enabling agents like agent bond.
Flexible file mappings allow multiple alternative templates.
Defaults allow shared configuration between multiple parameter values.

fish-pepper can be seen in its fully beauty in fabric8io/base-images where more than twenty five base images are maintained with fish-pepper.

With node.js you can install fish-pepper super easy with

npm -g install fish-pepper

In the following blogs I will show more usage examples, especially how “blocks” can be easily reused and shared.

Jmx4Perl for everyone

Tue, 28 Jul 2015 00:00:00 +0000

As you might know, Jmx4Perl is the mother of Jolokia. But what might be not so known is, that Jmx4Perl provides a set of nice CLI tools for accessing Jolokia agents. However, installing Jmx4Perl manually is cumbersome because of its many Perl and also native dependencies.

However, if you are a Docker user there is now a super easy way to benefit from this gems.

Even if Perl is not your cup of tea, you might like the following tool (for which of course no Perl knowledge is required at all):

jmx4perl is a command line tool for one-shot querying Jolokia agents. It is perfectly suited for shell scripts.
j4psh is a readline based, JMX shell with coloring and command line completion. You can navigate the JMX namespace like directories with cd and ls, read JMX attributes with cat and execute operations with exec.
jolokia is an agent management tool which helps you in downloading Jolokia agents of various types (war, jvm, osgi, mule) and versions. It also knows how to repackage agents e.g. for enabling security for the war agent by in-place modification of the web.xml descriptor.
check_jmx4perl is a full featured Nagios plugin.

How can you now use these tools ? All you need is a running Docker installation. The tools mentioned above are all included within the Docker image jolokia/jmx4perl which is available from Docker Hub.

Some examples:

# Get some basic information of the server
docker run --rm -it jolokia/jmx4perl \
 jmx4perl http://localhost:8080/jolokia

# Download the current jolokia.war agent
docker run --rm -it -v `pwd`:/jolokia jolokia/jmx4perl \
 jolokia

# Start an interactive JMX shell
# server "tomcat" is defined in ~/.j4p/jmx4perl.config
docker run --rm -it -v ~/.j4p:/root/.j4p jolokia/jmx4perl \
 j4psh tomcat

In these examples we mounted some volumes:

If you put your server definitions into ~/.j4p/jmx4perl.config you can use them by mounting this directory as volume with -v ~/.j4p:/root/.j4p.
For the management tool jolokia it is recommended to mount the local directory with -v $(pwd):/jolokia so that downloaded artefacts are stored in the current host directory. (Note for boot2docker users: This works only when you are in a directory below you home directory)

It is recommended to use aliases as abbreviations:

alias jmx4perl="docker run --rm -it -v ~/.j4p:/root/.j4p jolokia/jmx4perl jmx4perl"
alias jolokia="docker run --rm -it -v `pwd`:/jolokia jolokia/jmx4perl jolokia"
alias j4psh="docker run --rm -it -v ~/.j4p:/root/.j4p jolokia/jmx4perl j4psh"

As an additional benefit of using Jmx4Perl that way, you can access servers which are not directly reachable by you. The Jolokia agent must be reachable by the Docker daemon only. For example, you can communicate with a SSL secured Docker daemon running in a DMZ only. From there you can easily reach any other server with a Jolokia agent installed, so there is no need to open access to all servers from your local host directly.

Finally, here’s a short appetiser with an (older) demo showing j4psh in action.

docker:watch

Tue, 30 Jun 2015 00:00:00 +0000

Ok, you know Docker. And since you are a Java developer you want to know how you can use this in your daily development workflow. You probably also heard about the docker-maven-plugin which seamlessly creates Docker images, starts and stops Docker containers and more all with a concise configuration syntax.

And now there is this new goal docker:watch.

We developers are lazy, right ? We want our code to compile fast, we want the servers to start up fast. And we want to test changes quickly. That’s why we love OSGi¹ and JRebel. And we want this for Docker containers, too.

Good news. docker-maven-plugin will support hot rebuild of Docker images and hot restart of containers with a new Maven goal docker:watch. It will be released with version 0.12.1. For the brave coder 0.12.1-SNAPSHOT is already out there, the documentation can be found here.

But before losing more words, here’s a sneak preview.

That’s of course not entirely true. We love OSGi … or we hate it with passion. But even the haters don’t hate it for its hot deployment abilities. ↩︎

The Docker Wormhole Pattern

Wed, 17 Jun 2015 00:00:00 +0000

Building a Docker wormhole is easy.

A wormhole is a special type of structure that some scientists think might exist, connecting parts of space and time that are not usually connected

— Cambridge Dictionaries Online

In Docker universe we have several uses cases, which require a Docker installation within a Docker container. For example the OpenShift Builds use images whose container’s are meant to create application images. They include a whole development environment including possibly a compiler and a build tool. During the build a Docker daemon is accessed for creating the final application image.

The question is now, how a build can access the Docker daemon ? In general, there are two possibilities:

Docker in Docker is a project which allows you to run a Docker daemon within a Docker container. Technically this is quite tricky and there seems to be some issues with this approach, especially because you have to run this container in privileged mode. This is the Matryoshka doll pattern.
Or you use the Wormhole pattern described in this post. The idea is to get access to the Docker daemon running a container from within the container.

As you know a Docker host can be configured to be accessible by two alternative methods: Via a Unix socket or via a TCP socket.

Using the Unix socket of the surrounding docker daemon is easy: Simply share the path to the unix socket as a volume:

# Map local unix socket into the container
docker run -it -v /var/run/docker.sock:/var/run/docker.sock ...

Then within the container you can use the Docker CLI or any tool that uses the Unix socket at usual.

Running over the TCP socket is a bit more tricky because you have to find out the address of your Docker daemon host. This can best be done by examining the routing table within the container:

# Lookup and parse routing table 
host=$(ip route show 0.0.0.0/0 | \
 grep -Eo 'via \S+' | \
 awk '{print $2}');
export DOCKER_HOST=tcp://${host}:2375

This works fine as long you are not using SSL. With SSL in place you need have access to the SSL client certificates. Of course this is achieved again with a volume mount. Assuming that you are using boot2docker this could look like

# Mount certs into the container
docker run -ti -v ~/.boot2docker/certs/boot2docker-vm/:/certs ....

This will mount your certs at /certs within the container and can be used to set the DOCKER_HOST variable.

if [ -f /certs/key.pem ]; then
 # If certs are mounted, use SSL ...
 export DOCKER_CERT_PATH=/certs
 export DOCKER_TLS_VERIFY=1
 export DOCKER_HOST=tcp://${host}:2376
else
 # ... otherwise use plain http
 export DOCKER_TLS_VERIFY=0
 export DOCKER_HOST=tcp://${host}:2375
fi

There is some final gotcha as that the server certificate can not be verified because it doesn’t contain the docker host IP as seen from the container. See this issue for details. As workaround you have to unset DOCKER_TLS_VERIFY for the moment when using the docker client.

Both ways are useful and are leaner and possibly more secure than having a Matryoshka doll approach.

Finally there is still the question, why on earth wormhole pattern ? Like in a wormhole (also known as Einstein-Rosen Bridge) you can reach through the wormhole a point (the outer docker daemon) in spacetime which is normally not reachable (because a container is supposed to be its “own” world). Another fun fact: If you create a container through a wormhole this container it’s not your daughter, its your sister. Feels a bit freaky, or ? Alternatively you could call it also Münchhausen pattern because you create something with the exact the identically means you have been created yourself (like in the Münchhausen trilemma).

Or feel free to call it what you like ;-)

Real clean Maven builds with Docker

Fri, 07 Nov 2014 00:00:00 +0000

A local Maven repository serves as a cache for artifacts and dependencies, we all know this. This helps in speeding up things but can cause subtle problems when doing releases. Docker can help here a bit for avoiding caching issues.

Before doing a release I typically move ~/.m2/repository away to be really sure that everybody else can build the source as well and that any dependencies are also on the remote Maven repository. This is a bit tedious, because it is a manual process and you can forget to move the old directory back which will was a LOT of disk space over time.

Docker can help here a bit: Since yesterday there is an official Maven image which can be used to build your project. The nice thing for doing releases with this image is, that it always starts afresh with an empty local Maven repository.

Assuming you are currently located in the top-level directory holding your pom.xml you can use this single command for running a real clean build:

docker run -it --rm \ 
	 -v "$(pwd)":/usr/src/mymaven \ 
	 -w /usr/src/mymaven \ 
	 maven:3.2-jdk-7 \
	 mvn clean install

With this call you mount your project directory into /usr/src/mymaven on the container, change to this directory in the container and call mvn clean install. At the end, your container will be removed (--rm) so there is no chance that you might forget to clean up afterwards.

Of course it will download all the artifacts each time, so it is not a good idea to use this approach for your daily developer business (especially if you using Maven central as remote Maven repository).

You can also play around with various versions of Maven by changing the image tag so at the end you can be really sure, that your project will build everywhere. Please refer to the Docker Hub page for details.

Update: As pointed out by Noah Zucker on Twitter you can redirect of course the local repository via -Dmaven.repo.local=/tmp/clean-repo temporarily to a new location. Which is confessedly much simpler and I would prefer that one instead if you don’t need to check with different JDKs or Maven versions. Sometimes you don’t see the forest for the trees if you come from the wrong direction (e.g. looking for use case of a specific docker image).

Docker maven plugin rewrite

Mon, 13 Oct 2014 00:00:00 +0000

My docker-maven-plugin is undergoing a major refactoring. This post explains the motivation behind this and also what you can expect in the very near future. The configuration syntax becomes much cleaner and implicit behavior was removed.

Originally, I needed a docker-maven-plugin for a very specific use case: To test Jolokia, the HTTP-JMX bridge in all the JEE and non-JEE servers out there. This was a very manual process: Fire up the VirtualBox image with all those servers installed, start a server, deploy Jolokia, run integration tests, stop the server, start the next server …. It takes easily half a day or more to do the tests before each release. That’s not the kind of QA you are looking for, really. But with docker there is finally the opportunity to automate all this: Deploy a single application on multiple different servers while controlling the lifecycle of theses servers from within the build.

Early this year when I searched the Web, I couldn’t find a good Docker build integration, so I decided to write my own docker-maven-plugin to back my use case. Today you find nearly a dozen of Maven plugins for your Docker business. However, only four plugins (alexec, wouterd, spotify and rhuss) are still actively maintained. A later blog post will present a detailed comparison between those four plugins (or come to my W-JAX session), but this post is about the evolution of the rhuss plugin.

It turned out that the plugin works quite well, people liked and starred it on GitHub. It provides also some unique features like creating Docker images from assembly descriptors.

But I was not so happy.

The reason is, that I started from a very special, probably very uncommon use case: A single application, multiple different servers for multiples tests. A much more common scenario is to have a fixed application server brand for the application, running with multiple linked backend containers like databases. My plugin doesn’t work well with running multiple containers at once. Or to state it otherwise: The plugin was not prepared for orchestration of multiple docker containers.

Also, there was too much happening magically behind the scenes: When pushing a data image, it was implicitly build. When starting a container for integration test, the data container is also build before.

Two operational modes were supported: One with images holding the server and data separately in two containers (linked via volumes) and one so called merged image, holding both, the application and server together in one image. This is perfect for creating micro services. The mode is determined only by a configuration flag (mergeData), but it is not really clear how many and what Docker images are created. And it was hard to document which is always a very bad smell.

So I changed the configuration syntax completely.

It is now much more explicit and you will know merely by looking at the configuration which and how many containers will be started during integration testing and what the container with the application will look like. I don’t want to go into much detail here, the post is already too long. Instead here is an example of the new syntax:

<plugin>
 <groupId>org.jolokia</groupId>
 <artifactId>docker-maven-plugin</artifactId>
 <version>0.10.1</version>
 <configuration>
 <images>
 <image>
 <name>consol/tomcat-7.0</name>
 <run>
 <volumes>
 <from>jolokia/docker-jolokia-demo</from>
 </volumes>
 <ports>
 <port>jolokia.port:8080</port>
 </ports>
 <wait>
 <url>http://localhost:${jolokia.port}/jolokia</url>
 <time>10000</time>
 </wait>
 </run>
 </image>
 <image>
 <name>jolokia/docker-jolokia-demo</name>
 <build>
 <assemblyDescriptor>src/main/assembly.xml</assemblyDescriptor>
 </build>
 </image>
 </images>
 </configuration>
</plugin>

This examples creates and starts two containers during docker:start, linked together via the volumes directive. The <run> configuration section is used to describe the runtime behavior for docker:start and docker:stop, and <build> is for specifying how images are build up during docker:build.

Alternatively, a single image could be created:

<plugin>
 <groupId>org.jolokia</groupId>
 <artifactId>docker-maven-plugin</artifactId>
 <version>0.10.1</version>
 <configuration>
 <images>
 <image>
 <name>jolokia/docker-jolokia-combined-demo</name>
 <build>
 <baseImage>consol/tomcat-7.0</baseImage>
 <assemblyDescriptor>src/main/assembly.xml</assemblyDescriptor>
 </build>
 <run>
 <ports>
 <port>jolokia.port:8080</port>
 </ports>
 <wait>
 <url>http://localhost:${jolokia.port}/jolokia</url>
 </wait>
 </run>
 </image>
 </images>
 </configuration>
</plugin>

Here consol/tomcat-7.0 is used as base for the image to build and the data referenced in the assembly descriptor is copied into the image. So there is no need to volume-link them together.

I won’t repeat the old, more confusing syntax for this both use cases here, you find it in the current online documentation.

Said all that, and since rhuss/docker-maven-plugin is still pre-1.0, I take the liberty to change it without much thoughts on backwards compatibility (you can easily update old configurations). The new syntax is available since 0.10.1, the old syntax will still be used in the 0.9.x line. Everybody is encouraged to upgrade to 0.10.x, although the documentation still reflects the old syntax (will be fixed soon). Please refer to the examples on the new-config branch for more details. An upgrade path will be available soon, too.

There will be a 1.0.0 release before the end of this year.

Please let me know your feedback on the new syntax and what features you would like to see. Everything is moving before the 1.0.0 freeze. You can open an issue for any suggestion or feature request.

Spicy Docker Java Images with Jolokia

Thu, 09 Oct 2014 00:00:00 +0000

While on the way of transforming the Jolokia integration test suite from a tedious, manual, half-a-day procedure to a full automated process I ran into and felt in love with Docker. As a byproduct a java-jolokia docker repository emerged, which can be easily used as a Java base image for enabling a Jolokia JVM agent during startup for any Java application.

These images are variants of the official java Java docker image. In order to use the Jolokia agent, a child image should call the script jolokia_opts (which is in the path). This will echo all relevant startup options that should be included as argument to the Java startup command.

Here is a simple example for creating a Tomcat 7 images which starts Jolokia along with Tomcat:

FROM jolokia/java-jolokia:7
ENV TOMCAT_VERSION 7.0.55
ENV TC apache-tomcat-${TOMCAT_VERSION}

EXPOSE 8080 8778
RUN wget http://archive.apache.org/dist/tomcat/tomcat-7/v${TOMCAT_VERSION}/bin/${TC}.tar.gz
RUN tar xzf ${TC}.tar.gz -C /opt

CMD env CATALINA_OPTS=$(jolokia_opts) /opt/${TC}/bin/catalina.sh run

(Don’t forget to use $(jolokia_opts) or with backticks, but not ${jolokia_opts})

The configuration of the Jolokia agent can be influenced with various environments variables which can be given when starting the container:

JOLOKIA_OFF : If set disables activation of Jolokia. By default, Jolokia is enabled.
JOLOKIA_CONFIG : If set uses this file (including path) as Jolokia JVM agent properties (as described in Jolokia’s reference manual. By default this is /opt/jolokia/jolokia.properties. If this file exists, it will automatically be taken as configuration
JOLOKIA_HOST : Host address to bind to (Default: 0.0.0.0)
JOLOKIA_PORT : Port to use (Default: 8778)
JOLOKIA_USER : User for authentication. By default authentication is switched off.
JOLOKIA_PASSWORD : Password for authentication. By default authentication is switched off.

So, if you start your tomcat with docker run -e JOLOKIA_OFF no agent will be started.

Currently this image is available from Docker Hub for the latest versions of Java 6,7 and 8, respectively, as they are provided by the official Docker java image.

Other base images can be easily added by using the configuration and templates from a super simple node based build system.

All appserver images from ConSol/docker-appserver (Docker Hub) are based now on this image, so Jolokia will always be by your side ;-)

Using NSEnter with Boot2Docker

Mon, 01 Sep 2014 00:00:00 +0000

NSEnter is a nice way to connect to a running Docker container. This post presents a script to simplify the usage of nsenter together with Boot2Docker.

There is still quite some dust around Docker and after gaining more and more experience, new patterns and anti-patterns are emerging.

One of those anti-patterns is the usage of an SSH daemon inside an image for debugging, backup and troubleshooting purposes. Jérôme Petazzoni’s Blog Post explains this nicely. In addition it provides proper solutions for common use cases for which SSH is currently used.

Nevertheless I still have this irresistible urge to login into a container. And if it is only for looking around and checking out the environment (call me old-fashioned, that’s ok ;-)

Luckily Jérôme provides a perfect solution to satisfy this thirst: nsenter. This allows you to enter into container namespaces. On the GitHub page you find the corresponding recipe for installing and using nsenter on a Linux host.

If you want to use it from OS X with e.g. Boot2Docker you need to login into the VM hosting the Docker daemon and then connect to a running container.

As described in the NSenter README you can use a simple alias for doing this transparently

	docker-enter() {
	 boot2docker ssh '[ -f /var/lib/boot2docker/nsenter ] || docker run --rm -v /var/lib/boot2docker/:/target jpetazzo/nsenter'
	 boot2docker ssh -t sudo /var/lib/boot2docker/docker-enter "$@"
	}

For a bit more comfort with usage information and error checking you can convert this to a small shell script like docker-enter which needs to be installed within the path (on OS X). As arguments it expects a container id or name and optionally a command (with args) to execute in the container. This script also will automatically install nsenter on the boot2docker VM if not already present (like the shell function above does this as well):

	10:20 [~] $ docker ps -q
	5bf8a161cceb
	
	10:20 [~] $ docker-enter 5bf8a161cceb bash
	
	Unable to find image 'jpetazzo/nsenter' locally
	Pulling repository jpetazzo/nsenter
	Installing nsenter to /target
	Installing docker-enter to /target
	
	root@5bf8a161cceb:/#

If you want even more comfort with bash completion you can add the small Bash completion script docker-enter_commands (inspired by and copied from Docker’s bash completion) to your ~/.bash_completion_scripts/ directory (or wherever your completion scripts are located, e.g. /usr/local/etc/bash_completion.d if you installed bash-completion via brew). This setup completes on container names and ids on the arguments for docker-enter. Alternatively you can put the commands together with the shell function code above directly into your ~/.bashrc, too.

P.S. After writing this post, I’ve found out, that this topic has been already covered in another blog post previously by Lajos Papp. That’s also where the shell function definition in the nsenter README originates from. Give credit to whom it’s due.

Docker for (Java) Developers

Mon, 25 Aug 2014 00:00:00 +0000

Recently I gave a Meetup talk for the Docker Munich Meetup Group which explained how Docker can help developers to improve integration tests and to ship applications.

The slides are online as well as the demo project.

During the demo I used Butterfly for an in-browser shell, which was quite cool, I guess ;-) (This is obviously not enabled in the online slides).

I’m going to continue to celebrate my Docker-♡ with another two talks in autumn:

Boosting your developer toolbox with Docker at JBoss One Day Talk, September, 29. in Germering (Munich, Germany)
Docker für Java Entwickler (in german) at W-JAX 14, November, 3. - 7. in Munich

And there is a slight chance (since the CFP has not yet been declined ;-) to talk at Devoxx about Docker. JavaZone unfortunately declined my CFP, I guess there are already too many riding the docker horse (which is a good thing).

Nevertheless I will attend both conferences with talks about Jolokia, and I’m really looking forward to it.

‘guess it will become a hot autumn (hotter than this german 2014 summer for sure) ….