Docker, Kubernetes, and other technologies for containerized applications are the must of modern development, especially in the case of microservices. But containerization is an intricate science, and even seasoned developers can make mistakes when working with containers. In this article, you will find out how to avoid these pitfalls and solve the issues you might have already encountered.
Table of Contents
Overview
The process of containerization seems easy — pack the application in a container, deploy it to the cloud, done. But later you run against decreased performance and increased cloud bills: What could go wrong?
There are typical mistakes in Java development leading to deteriorated performance, including
- Faulty application logic;
- Incorrect usage of databases;
- Concurrency issues;
- RAM under- or overutilization;
- Improper server infrastructure.
In the case of containers, developers should focus on memory usage and infrastructure. Below you will find the recommendations on how to customize the containers and tune JVM settings to reach optimal performance and footprint indicators. But before we go any further, it is worth noting that no JVM fine-tuning will help eliminate strategic errors. Just like local optimization in algorithms cannot beat asymptotics, container tuning won’t fix the code with memory leaks or unnecessary calculations. So don’t neglect application profiling to prevent or remedy such errors on time.
Be cautious about automatic Docker image generation
Imagine you already have a well-written Java application and want to spend minimum time and effort to containerize it and deploy into production. There already exist solutions in the Java world that help with automatic container generation, for example, Paketo Buildpacks. Simply install the pack utility and then follow the official guidelines on developing applications with this tool. Alternatively, check our guide on utilizing buildpacks for Java workloads.
Paketo Buildpacks use Liberica JDK, a Java runtime recommended by Spring, by default. It provides JDK at build time and JRE at runtime, so the resulting container will be smaller. But despite the fact that buildpacks accelerate and facilitate development, neglecting the JVM tuning will lead to deteriorated performance.
For instance, if you run the application built automatically, you may notice that it is using less CPU and cores than allocated in the container. For instance, the container can work with 4 GB of memory and 16 processors, but the JVM has access only to 1 GB and 2 threads.
This particular issue was solved in the latest buildpack versions. It can also be resolved by setting the container parameters manually and turning off automatic core count and memory calculator with
docker run --rm --tty --publish 8080:8080 --env 'JAVA_TOOL_OPTIONS=-Xmx3072m -XX:ActiveProcessorCount=4' --memory=4G paketo-demo-appWe recommend always checking and setting these parameters correctly. Automatic image generation saves time, but as a result you get a black box that may not fit your purposes. Luckily, there are numerous configuration options you can implement to tune the JVM at runtime. The full list of flags can be found on the Liberica Buildpack page.
Set the -Xmx parameter correctly
If you want to create the image of your application manually, you need to set the -Xmx parameter correctly. It is used to define the max. heap size, but to state the right number you should first define how much RAM the application uses.
The idea is to set the RAM limit for the container higher than the JVM heap size. In addition, the server should have enough RAM to start all your containers.
If the -Xmx value is less than the application needs, it will crash with the OutOfMemoryError: Java heap space error.
How to determine the amount of memory your application requires? The easiest method is to activate the GC logging in JVM parameters by running
-verbose:gc -XX:+PrintGCDetails
This way, before the application exits, the total and used memory values will be printed in the console.
The amount of memory allocated through -Xmx should be bigger than the de facto used memory found in the GC logs. The difference depends on the peak values. For that purpose, perform load testing, run the application with a profiler, for instance, Java Flight Recorder, and analyze how RAM consumption depends on the load.
Avoid automatic SerialGC switching
Java has a remarkable feature: if you limit your application to 2 GB RAM and less than 2 processors, SerialGC will switch automatically.
SerialGC is the oldest and relatively efficient garbage collector perfect for single-thread applications. But at the same time, it is the slowest Java collector.
The conditions of SerialGC activation are detailed in the select_gc_ergonomically() and is_server_class_machine() functions in OpenJDK documentation. Note that flags such as -XX:+UseG1GC won’t help if you allocate too little memory.
To sum up, even if you want to save memory, do not set less than 2 GB in the -Xmx parameter. To further enhance garbage collection, choose a GC implementation best fit for your workloads.
Use newer OpenJDK distributions
The OpenJDK community has been working actively on container support. A lot of issues have been eliminated.
For example, there is a known issue: top and free tools inside of a container show the total host memory, not the one you assigned in the -memory parameter. You can check it by running the container in the interactive mode:
docker run \
--interactive \
--tty \
--memory 10m \
bellsoft/liberica-openjdk-alpine
After that, you can run commands such as free -m or top bn1 in the console. No matter how you experiment with startup flags, you won’t see the desired 10 MB. The console will print only the total amount of memory on your computer.
Earlier, JVM demonstrated the same behavior. Run this command in the console:
docker run -m 1gb openjdk:8u131 java -XshowSettings:vm -version
Now compare it to the latest OpenJDK version:
docker run -m 1gb liberica-openjdk-alpine:latest java -XshowSettings:vm -version
When using the older version, the console output will be the total host memory just like with free. In the case of the newer version, the value will be equal to about a quarter of a claimed memory limit. The same goes for processor count:
Runtime.getRuntime().availableProcessors()It is the result of introducing the -XX:+UseContainerSupport flag into OpenJDK 10 and further, where it is activated by default.
You can experiment with this flag. Switch it off, and you will get the previous behavior even with the newer OpenJDK versions:
docker run -m 1gb liberica-openjdk-alpine:latest java -XX:-UseContainerSupport -XshowSettings:vm -version
It means that containers are accounted for in the latest versions. Therefore, you should always update your distribution to the latest version to prevent similar issues.
You don’t have to migrate straight to JDK 17, the latest LTS release, although it is desirable because this version contains a lot of new features. If you work with Java 8, this problem was fixed in 8u212. But the best practice is to use the newest official OpenJDK release for your Java version. Liberica JDK updates always come out on time guaranteeing that your runtime will be free of known bugs and vulnerabilities.
Set memory limits for the container
Containers also have memory limitations. For example, you can set the memory limit in Docker with the -memory flag. Kubernetes has a more complex system: you have to specify request (searches for the most suitable server) and limit (sets strict memory limits for cgroups) separately. In any case, it makes sense to allocate more memory to the container than the app requires, but there are certain intricacies we will discuss below.
How to enable Native Memory Tracking
GC logs don’t show all the memory used by your application. For example, extra resources may be spent when utilizing
- Off-heap memory (
ByteBuffer.allocateDirect); - External native libraries loaded through
System.loadLibrary; - Significant heap fragmentation due to fact that malloc allocates memory in blocks;
and so on.
When determining container memory limits, you should consider not only- Xmx, but also total memory. You can check it by using Native Memory Tracking.
First, start your Java application with the following flag:
-XX:NativeMemoryTracking=detail
Then, to see the total memory with its internal part, run
jcmd $pid VM.native_memory
where $pid is the Java process identifier. It can be found by running jcmd without parameters or using ps aux | grep java.
You can track memory changes by using
jcmd $pid VM.native_memory detail.diff
How to disable Swap Memory
Suppose you already know the precise amount of memory required by your application. It is possible to set the -Xmx value higher than available memory on the server or in the container, but the performance will deteriorate.
The reason is that heaps in Java differ from those in C/C++. Scripts and native programs in C/C++ efficiently make it into the swap. In Java, we have to work with one heap, where Java objects are evenly distributed over the whole address space of the process. If most of them get into the swap file, the application will slow down significantly.
What should we do to avoid that? Set the correct -Xmx value or even disable swap in the host machine’s OS and the container. Use a swapoff -a command for host and delete the respective entries from the /etc/fstab file (sed -i '/ swap / s/^/#/' /etc/fstab). In Docker, set the --memory-swap parameter equal to --memory.
Fortunately, Kubelet will not work with enabled swap memory by default if you didn’t allow it explicitly with KUBELET_EXTRA_ARGS=--fail-swap-on=false and the memorySwap parameter in KubeletConfiguration.
Choose a small base image
One more important factor to consider is the size of the resulting container. If you stuff it with unnecessary files, OS packages, dependencies, JDK instead of JRE, and a heavy base OS image, you will soon notice how the bills for cloud storage start to inflate.
We prepared a detailed guide on trimming your Docker container images of Java applications, which will help you master useful Docker commands and Java tools to reduce the size of Java containers.
But in many cases, implementing the right OS enables the developers to cut the size of the container image instantly and without other adjustments. The question is, which OS to choose? CentOS is no longer supported and filled with vulnerabilities, RHEL doesn’t have a free version, Alpine Linux, albeit small, lacks commercial support.
Seeing the demand for small, secure, and supported Linux distro, BellSoft created Alpaquita Linux — a 100% Alpine-compatible distribution with the base image size of 3.32 Mb (musl) and 8.67 Mb (glibc). Apart from the small size, Alpaquita boasts several distinguishing features:
- Two libc implementations, optimized musl and glibc;
- Security hardening;
- LTS releases;
- Tools facilitating Java development and four mallocs for various Java workloads.
You can use containers with Alpaquita Linux and Liberica JDK Lite for free. The migration is extremely easy and requires changing only one line in your Dockerfile:
FROM bellsoft/liberica-runtime-container:jdk-17-stream-musl as builderAlternatively, browse the Liberica Runtime Container repository for the most suitable image.
In any case, you will get a lightweight container with the latest security patches both for JDK and Linux from one vendor as a result. For Spring Boot developers, there’s more — they will be able to save up to 30% RAM immediately after switching to Alpaquita containers!
Conclusion
To sum up, successful containerization rests upon the following principles:
- Determine how much memory your application requires;
- Set the -Xmx parameter and RAM limits for the container accordingly;
- Use the latest OpenJDK builds for your Java version;
- Tune the system, for example, by disabling swap.
This article explains the fundamentals of the processes mentioned above. If you want to make a deeper dive into Java containerization practices, check out these materials:
