Analyzing Docker’s New OSS: libchan and libswarm

This morning’s keynote from Solomon Hykes was inspiring. Just after the keynote, Solomon told me he was really nervous. But if he was, it didn’t show. The way they are building out their ecosystem is as equally inspiring as Solomon’s speech this morning.

Today, there were three new projects announced from Docker:

And two “secret” Docker projects were hinted at too:

  • identity
  • authentication

This is really exciting news for people building tools around Docker, however it may be less interesting to people who are just using Docker for building apps.

BpyF0uMCIAAedgn

Review the News

Let’s review what this means for the Docker ecosystem:

swarm-honeybees-maine-1074446-xl

libswarm

In Docker’s words: Libswarm is a toolkit for composing network services

In my words: Libswarm is like an ORM or a type system for distributed systems. The biggest problem with Docker is multi-container and multi-server inter-container communication.

What does that mean without buzz words? Imagine being able to run docker ps and it shows all your containers running on 5 different infrastructures (some public, some private, some on your laptop). That’s what libswarm enables… a swarm of Docker containers available through a unified API.

Here is a demo video shown at the DockerCon Keynote made by Ben Firshman of Orchard of what you can do with libswarm:

The “Distributed Docker Containers” problem has lead to lot of open-source projects on top of Docker:

  • etcd
  • fleet
  • geard
  • mesos
  • shipyard
  • serf
  • and more…

The problem with so many choices is that they all have their own protocols and work in different ways. Once you adopt one of these tools, you effectively are locked into their solution because they don’t share an API or an underlying communication method.

libswarm exposes a standard interface to systems like Fleet, Mesos, Consul etc

Libswarm is built to help provide a shared API between all of these orchestration tools. This is a brilliant move from the Docker guys. It simultaneously creates a common ground that allows people to collaborate on orchestration, but it also encourages healthy competition.

When the orchestration tools are interoperable, it breaks vendor lock-in which allows people to move more freely back and forth and lets the best tool win. It levels the playing ground. This is a revolutionary building block for Docker ecosystem development. It keeps incumbents honest and gives up-and-comers a real chance to create something new and better.

Solomon notes on Hacker News that this project is very rough and early, and not yet ready for production use yet.

libchan

Like Go channels over the network

In Docker’s words: Libchan is an ultra-lightweight networking library which lets network services communicate in the same way that goroutines communicate using channels.

It is for:

  • Simple message passing
  • Synchronization for concurrent programming
  • Nesting: channels can send channels

In my words: Libchan is the underlying library that enables libswarm. It is like ZeroMQ or RabbitMQ for Docker. The difference is that the existing messaging tools are heavyweight and general purpose… libchan is built to support Docker and containerized apps in Docker.

For network engineers and people who get excited about Unix sockets, this tool is very exciting. For an average developer, you will more likely interact with things built on libswarm than libchan.

libcontainer

This was not the biggest news of the day, since libcontainer has been a part of Docker for a while now. It is simply spinning it out into its own repository. However it is a smart move and it will enable faster velocity in expanding support for more types of containers and more abstract support for things like managing VMs.

Conclusions

The Docker community continues to evolve quickly and tackle big challenges. The new projects announced this morning will go a long way to addressing some of the biggest problems Docker has to get to mainstream adoption.

  • http://ahmetalpbalkan.com/ Ahmet Alp Balkan

    Great summary, thanks.

  • http://grzegorzgraczyk.pqstudio.pl/ Grzegorz Graczyk

    Nice article. Love to read more about libswarm! :)