Monitoring Docker Containers – All you need to know

IMG_4101The use of container technologies such as Docker and Kebernetes and microsevices are on the rise as dev teams are seeking to innovate with faster release cycles. Continuous delivery is the name of the game as DevOps are looking to ship and iterate quicker and quicker. The rise of the popularity in container use shouldn’t come as much of a surprise and their flexibility is a key asset. However, container use comes with its own set of issues. One of those key issues is the lack of visibility into the containers themselves, which results in operational blind spots and even more places where things can go awry, amid the difficulties to identify performance issues.

The ability to scale horizontally becomes easier when containers are decoupled into multiple ones. On top of this, the containers become reusable. A web application stack for instance, can be comprised of three individual containers, each having their own unique image. This allows for the stack to handle management of the web application itself, database as well as an in-memory cache, all decoupled.

Applications can be run in an isolated environment when using Dockerized containers.  Lacking the overhead of a virtual machine, the containers share the same kernel and use cgroups and namespaces. Docker facilitates decoupling, allowing it to scale horizontally and reuse the containers. Docker makes the decoupling easier while it isolates the processes within containers.

With AIOps bursting onto the ITOps scene, you have a multi layered technology platform that automates and greatly enhances IT Operations via the use of machine learning and analytics to analyze the big data emanating from tools and devices. It therefore has the capability to identify and alert personnel of issues in real time. In some cases, an AIOps platform can remedy these issues immediately without the need for human intervention.

DevOps teams cannot afford product bugs that affect performance, as this has a direct effect on the user experience, and most importantly the bottom line. What DevOps needs is automatic discovery of both entry and exit points of microservices for keeping microservices monitoring as tight as can be.

Tracking KPIs of your microservices is hard enough without needing to worry about the entire distributed business transaction that utilizes it. Going further, effective tracking allows you to drill down to isolate performance issue root causes.

DockerHow to get started with Monitoring Docker Containers

The Docker craze has empowered product teams with freedom in their choice of technologies as they now can deploy and manage their application in production on their own. With all the benefits of utilizing Dockers, comes the complexity, as putting Dockers into the operational mix exists a wealth of application and infrastructure data. So also comes the greater need to monitor the production environment and alerting those needed when issues arise.

Below I’ll go over the questions you need to ask and the steps you need to take to get started with monitoring docker containers. This includes collecting the container metrics you most care about as well as the options available to you in collecting application metrics.

As engineering, operations and IT all have huddled around the use and value of using containers, a solitary question endures. How does an organization monitor Docker in a production environment? A lot of this confusion is because it’s basically the wrong question that needs to be asked. See, monitoring Docker daemons or the Kubernetes master isn’t such a big deal. While, it is all required, the solutions are there.

It comes down to this. Using Docker containers to run your applications only modifies how they’re packaged, scheduled and orchestrated. Docker containers do not change the way your applications run. The question now becomes “how does Docker shift how you monitor your applications?”

As so often, the answer to this depends on a few factors. It depends on your environment’s dependencies which is affected by your use case as well as objectives.

Some of the questions you need to answer before getting started with a solution for monitoring docker containers are:

  • What’s your Docker image philosophy?
  • What orchestration technology is in place?
  • What’s the observation level you’re getting from your dockerized application(s)?

Going further, to truly understands how implementing a dockerized environment and a microservices strategy can play into your monitoring strategies, these basic questions should be answered.

  • Will you be tracking metrics that are application specific or just system-level metrics?
  • Do you run a stripped-down single-process container or do you use heavy images with supervisord?
  • Is your application placement dynamic or static?  (i.e.  Do you utilize dynamic container placement, binpacking or scheduling or do you statically map what runs where)?

The Challenge of Monitoring Docker Containers

Back in 2013, the release of Docker signified a shift of gargantuan proportions as to how the software industry aspired to package as well as deploy their applications. Lots of companies threw their hats into the ring and a great many supporting technologies have sprouted since. The hype surrounding the container space as you can imagine has been monumental. Following we’ll aim to make sense of this confusion by explaining how containers are used within enterprises.

The rapid adoption of container use by the software industry tied to the associated goal of building microservices is causing a paradigm shift in the monitoring arena. The challenge is daunting for traditional monitoring solutions which are not accustomed to the granular functionality of today’s highly resilient and scalable applications. For example, if a solitary microservices component fails, you might not see a business impact, so an alert’s severity should match this. The up or down approach of most traditional monitoring tools come up short, and with that a lot of organizations are developing their own monitoring solutions in house.

With the rapid increase in the amount of services running within the application system, along with any underlying infrastructure components, a plethora of data is being generated. This BIG DATA problem might have a negative effect on traditional monitoring solutions, and what is ultimately needed is an AIOps type solution that would be able to apply the new rules of the game to data. This type of solution would empower teams by providing clear insights to make decisions on the fly as well as harness the power of data to recommend actions to solve issues.

The Solution to Monitoring Docker containers

In a general nutshell (in itself a container of sorts), a monitoring strategy you should strive to implement is one that would provide you with powerful reporting tied to three key questions:

  • What happened?
  • Who are the affected parties?
  • How is the issue fixed?

The aforementioned are key components of effective root-cause analysis as they help organizations identify the issue, where it may exist, its severity (are internal customers affected? external?) and give suggestions on how to resolve the issue quickly and in a determined fashion.

Monitoring docker containers effectively is best accomplished:

  • Where individual components are monitored as well as the system’s overall availability and behavior
  • When metrics are tied to application performance, response accuracy and security (data, users and company)
  • Powerful visualizations via at-a-glance dashboards
  • Effective log data collaboration where everyone has access

In conclusion, monitoring docker containers effectively should include the monitoring of the full stack of the software application. This translates to the monitoring of virtual as well as physical servers, and cloud services. Now, only just docker containers need to be added.

About the author: Boris Krasniansky is a Solution Architect at Correlsense.