Is containerization the future of Java development? Discover the answer in 2021 Containers Trend Report: Download now!

How code becomes a microservice

How code becomes a microservice


Published November 27, 2020


Java VM “vs” Native Image

We are continuing our three-part series on microservice architecture. This second article, as promised, is covering a narrower subject: the evolution of core technologies turning code into a working microservice. Also, we are going to see differences between runtime+bycode and pure native image approaches to building microservices. I’ll go over the various benefits they provide, examine the limitations they pose, and settle the argument once and for all! While this choice is usually one-and-done, your organization likely has to make many others every day. Instead of pouring hundreds of valuable person-hours into solving technical issues, why not trust experts in their field? Talk to a senior engineer at BellSoft to learn which solutions suit you best.

Systems for business needs

Modern software processes a lot of data and is accessed by many users. In a rapidly changing world, developers also rapidly change it, literally responding to business challenges of a minute. Java Virtual Machine (JVM) with Java and other JVM languages found a way to the hearts of backend developers with great programming frameworks and IDEs that do the heavy lifting when writing and running code.

The code we wrote before

In my first article in this series, I discussed many instances where data decomposition and improved functionality make life easier. Microservices are not as unique here.

The service-oriented architecture (SOA) was a solution for horizontal scalability in the early days. However, the required level of abstraction was too high, making the code SOA-oriented instead of domain-oriented. Along with the development practices evolving and hardware and networks speeding up, new and more lightweight communication methods appeared — such as WebServices and REST.

From a Java language perspective, it all started with systems based on inheritance and extra configuration. Early Java EE (more than two decades ago!) introduced servlets. They extend HttpServlet and are referenced in web.xml; as well as EJB beans that implement appropriate interfaces, they are also configured separately. Soon afterward, we saw configuration closer to code, e.g. when some code and XML were generated from javadoc annotations. Then real annotations appeared, and generics as well. Serialization has been there from the very beginning. New language features were used in later versions of Java EE specification to describe data and organize interaction between beans.

Although Java EE also specifies protocols like JNDI, a huge step for the developer is that it’s focused on the easy declaration of things actually managed by an application server. On the contrary, the representation of data for consumers different from similar beans was unmanaged. During the transition from Web 1.0 to Web 2.0, backends passed representation responsibility to frontends. But what is now the difference between an external request and a request from another part of the same distributed system? It disappeared.

Container dependency injection

Some time has passed, and metaprogramming with annotations has become a routine practice for Java developers. This is also true for framework developers. It’s worth mentioning that other languages provide good alternatives: like Kotlin trailing lambdas used for routing description in Ktor; and data classes in other languages that are being chased by new records in Java 14 (tools like Lombok provide a solution for older versions).

Annotations or various means of external configuration make it possible to animate plain class fields. Containers or frameworks that have control over class/bean lifecycle can inject necessary fields after constructing them in a sophisticated manner. Original code won’t see that complexity and won’t notice a replacement for the purpose of testing or a different way of communication. Modern IDEs well support this way of programming. It helps to write and test smaller components and thus to build more flexible and reliable software.

Such a pattern of building component models is called Contexts and Dependency Injection (CDI). It has been standardized in JSRs 299/365 with a reference implementation called Weld, which still lives in containers such as WildFly. And the underlying principle of such architecture is called Inversion of Control (IoC).

There are many tasks that probably shouldn’t be governed by strict standards. Nevertheless, they must be solved by frameworks. That is true for modules that connect to databases or help to build interactive HTML. In a particular framework, the set of modules is limited, so it is possible to create an effective programming model focused on developer productivity. IoC turned out to be a good match for shaping such frameworks. The most famous example is the Spring Framework, which also follows one more useful paradigm called Convention over Configuration (also known as coding by convention).

Dynamic proxies

When we build loosely coupled systems, the role of reflection is critical. It powers discovery mechanisms, and components/beans find each other in run time, taking only required actions. One alternative is to be aware of all the requirements in advance and generate code that considers all planned interactions. Another is to generate and reload some classes on the fly. In any case, it’s important to have type-safe dependency injection.

In Java, we can create a facade instance that intercepts interface method calls. This design pattern is called a Dynamic Proxy and is made with the help of java.lang.reflect.Proxy and java.lang.reflect.InvocationHandler. This is a built-in reflection API mechanism, widely used in CDI containers, e.g. in Spring.

Remember the end goal

Today we are able to easily write tiny code snippets to weave remote systems such as other microservices, data, or messaging sources and respond to clients in ways they’ll understand.

@RestController
public class HelloController {

   @Autowired
   private WebClient client

   @RequestMapping(path = "/", method = RequestMethod.GET)
   public CompletableFuture<String> greet(Principal principal) {
       return client.get()
        .uri("http://api/persons/{id}", principal.getName())
        .accept(MediaType.APPLICATION_JSON)
        .exchange()
        .flatMap(response -> response.bodyToMono(Person.class))
        .map(person -> "Hello, " + person.getFirstName())
        .toFuture();
   }

}

This example here uses an id resolved after authentication to query person details from another service and respond with a greeting. Note that everything is encoded in a reactive style and expected to work asynchronously. The chosen framework and additional components such as service registry work powerful magic of service address discovery, load balancing, etc.

The role of a web server

Frameworks care about plugging in tunable components on a high level for the programmer. And yet, there is a world of lower-level communication protocols like HTTP where we need an intermediary between network connections and business logic. There’s also a need to manage some state (contexts), resources, security. That’s what web servers do. Formerly they managed multiple applications and required separate configuration. Now, in a time of containers, each microservice is coupled with its own server instance configured in a centralized manner with the framework’s help, like Embedded Tomcat.

A novel paradigm called Serverless does not actually exclude a web server. Just withdrawing server configuration and deployment from teams, this routine becomes backed by software as a service.

The role of JVM

JVM and the core class library serve as a foundation for the web server, libraries, and business logic, providing a rich set of means to make magic happen. They include network, threads & synchronization, dynamic proxies, classloaders… Well, everything that’s available in specifications serves microservice needs. Developer’s bytecode is verified and executed by the runtime and may be easily examined by standard diagnostic tools like Mission Control.

The way how machine resources are used to execute the code varies. Important JVM components such as just-in-time compilers and garbage collectors are flexible. It is possible to replace or disable a compiler or GC algorithm with a single command-line parameter.

There are also fine-tuning options that uncover all the power of JIT, GC, and other VM components when defaults are optimized for the actual hardware and OS limits where the service runs are still not good enough. And what does that mean? Each application has requirements for performance (like throughput and latency of processing) and first response time, along with resource limitations. Consumption of resources is related to costs. Examples here are deployment traffic, host memory and CPU utilization, disk usage. Bloated resource requirements equal waste. In the case of low performance, it can either mean waste or make a service considered entirely non-functional when it doesn’t meet service level agreements.

JVM tuning is bound to the tuning of the web server, framework, libraries, and the application as well. As they typically allow customization, it’s quite natural to configure them all at once. When it is possible to satisfy SLAs for some group of services, generalizing configuration practices is rather typical: I can think of a situation where a team mostly relies on a single GC algorithm.

After all (and especially after testing), JVM may be updated independently of business logic. So security updates, new features, and optimized defaults appear on production with the main code left intact and no rebuilds. Apart from just affecting JVM’s behavior, new features may be picked up on startup (probably with some configuration) by frameworks and web servers to execute more effectively or securely. That’s what we see when garbage collectors start to return memory to the OS, or when strings become compact, or when TLS 1.3 becomes available. And that’s what we will see when records and lightweight threads will recharge how data classes and connections work.

Frameworks

Above I’ve mentioned why we need frameworks so bad: they are critical to implementing many routine tasks typical for the plethora of projects. And just as these tasks and projects are numerous, frameworks themselves are so very abundant in our industry. There are both “heavyweights” and small niche ones. In the table below, we will list a few that we find the most widespread and pertinent. Choose any and test for yourself.

As of November 2020, all of the presented frameworks are supported and updated.

Name Vendor Initial release Languages (platform) Characteristics
Spring Pivotal Software 2002 Java, Kotlin, Groovy, dynamic languages (Jakarta EE*) Open source. The most popular among Java developers. Provides AOP capabilities, containerization, and integration with other frameworks.
MicroProfile Eclipse Foundation 2016 Java (Jakarta EE) A specification implemented within web servers and other frameworks. Aimed at optimizing Enterprise Java for the microservices architecture.
Quarkus Red Hat 2019 Java, Kotlin (Jakarta EE) Open source. Kubernetes-native, tailored for OpenJDK HotSpot and GraalVM. Uses JAX-RS.
Micronaut Object Computing, Inc. (OCI) 2020 Java, Groovy, Kotlin Open source. A JVM-based, full-stack framework for building modular microservice and serverless applications.
Helidon Oracle 2018 Java Open source. A collection of Java libraries for microservices-based apps running on a Netty web core. Supports MicroProfile. Uses JAX-RS.
Lagom Lightbend 2016 Java, Scala Open source. For building systems of Reactive microservices. Builds on Akka and Play.
Dropwizard Yammer Inc. 2011 Java, Scala Open source. Gathers popular libraries for lightweight packaging, such as Jetty, Jersey, Jackson, JUnit, and Guava.

* Former Java EE is now evolved to Jakarta EE developed by Eclipse Foundation.

Frameworks don’t exist in a vacuum: they are interconnected. Among other things, Eclipse MicroProfile addresses microservice architectures. It is not uncommon for frameworks to support multiple APIs because it allows to port existing code as is or with minor changes. You can use the CDI specification in Spring and Quarkus frameworks, which have their own means for the same purpose, or enable Spring annotations in Quarkus. Legacy systems are at fault here; when an organization switches frameworks, it’s ridiculous to make developers write new code from scratch. That’s why frameworks choose to adopt and support each other’s features.

Run it all together

So at some point, we have all parts of a microservice defined: JVM, web server, frameworks, libraries. Target OS, system packages, and hardware are also determined. Those assembled parts run in different environments. Developers and unit tests mock many software aspects and operate in unified surroundings; integration tests require a more production-like setup, then staging and production itself where a service interacts with numerous external entities.

Containerization is a modern approach to make all OS and external communications abstract without paying a high price, as is the case for hardware virtualization. Container management systems like Docker or Podman let us define, publish, and run container images with various software. Operating systems provide necessary levels of isolation and constrain resources for each container instance as requested by a management tool.

Software parts in a container image form a stack naturally. A nice feature of images is that the software stack described in terms of image levels makes physical deployment more effective. When top levels are updated, we may reuse cached lower ones. Thus, binaries from cached layers remain untransferred, and other preliminary actions do not perform. A layered image looks like this:

Microservice container layers

All these parts also need to be configured to interact with one another, work correctly in a host OS, and communicate with external systems. Developers and DevOps now have to deal with lots of configuration described in YAML that may contain lines, such as the following:

server:
  port : 8081
eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8761/eureka/

Finally, there are orchestration systems that make it possible to distribute containers over a cluster of machines, like Kubernetes and Marathon. At this point, traces of Java code and JVM disappear, building blocks are the configured containers deployed over the server fleet, and they may coexist with SaaS components available in a cloud.

Battle for resources

Now that we have our microservice running, configured, and distributed, another burning question comes up. How to minimize its consumption? I don’t need to tell you why it is important: as the organization grows, effective resource management often equals more performant products and more funds that can be allocated elsewhere.

When working on a project, the developer might encounter various points of potential bloating:

  1. Docker image size,
  2. memory footprint,
  3. startup time.

A containerized project requires access to a Java runtime. But here’s a deal: you’re not allowed to put Oracle JDK to your Docker container without having a license due to their policy. Instead, your company may freely use base images provided by BellSoft of about 107 MB (plus an option of 41.5 MB for CLI-like applications, which is the smallest container on the market!)

Running even a relatively little application will consume much memory. We’ve run an experiment by launching a differently configured example project.

JDK 15

  • 135 MB memory usage, started up in 4.395 sec without optimizations.
  • 70 MB memory usage, started up in 1.787 sec with thin jar and class sharing.

JDK 11

  • 165 MB memory usage, started up in 4.95 sec without optimizations.
  • 70 MB memory usage, started up in 2.034 sec with thin jar and class sharing.

Class loading slows down the startup. With thin jar and AppCDS, the speed has grown almost 2.5-fold.

Bottom line: we have a lightweight container image consuming 128 MB of memory without swap and starting up in 8 seconds. Compared to the original results, the startup has accelerated 2.5-fold, from 20 sec to 8 sec, and the memory footprint is reduced to one-sixth.

We’ve sacrificed peak performance for six times fewer memory requirements and startup under 10 seconds. After optimizing, compressing images by hundreds of megabytes, and speeding up the launch: Is it enough? I’m afraid not. Even with all Java’s might, JVM cannot go beyond its capabilities. And here’s where we turn to the Native Image approach.

Native Image

When assembled in a native image, the project used 35 MB in RAM and started up in 0.111 sec! At that, the native image itself is 89 MB.

It’s really hard to modify the code itself. However, trying a different platform to run it seems like an option, especially since unit and integration tests are available and help, say, migrate to newer versions of Java. Developers can also check additional assumptions about the code; finally, the build process can be altered relatively easily.

All of the above applies to changing service binary form to a native executable. Notable technology that helps here is GraalVM Native Image. A mix of Graal compiler applied in AOT mode, and SubstrateVM virtual machine promises instant startup time, low memory consumption, short disk space requirements, and excellent performance. For tools like that, ideally, we’d expect adding few dependencies, providing just one additional step to the build phase, and then getting every benefit by extending build scripts with something like:

gradlew nativeImage

What we got

The native image runs a Java microservice in closed-world assumptions. In its nature, this is simply a native executable, so it holds some interesting applications for container deployment. Base container level must provide very minimal functionality (and there’s no JDK), which means it may be a “scratch” base image. It will work if a deployed application brings its dependencies: e.g., a native image may be statically linked with a standard C library.

What is also typical is to require some more basic artifacts such as certificates, SSL libraries, and a dynamically loaded C library. In such a case, the binary may be linked in a different way, and the underlying base image will be a kind of “distroless” image. The distroless base image without libc by gcr.io is about 2 MB, which is slightly less than Alpine Linux, plus there is an option to install glibc, which gives a 17 MB base.

Comparing native image with thin and fat jars:

  Disk usage RAM usage Startup
Thin jar unoptimized 13 kb thin jar + 17.4 MB libs + 107 MB base image* 135 MB 2.197 sec
Thin jar optimized 13 kb jar + 17.4 MB libs + 107 MB base image + 50 MB jsa (CDS archive) 70 MB 1.156 sec
Fat jar unoptimized 18.02 MB jar + 107 MB base image 135 MB 3.811 sec
Native image 89.22 MB 35 MB 0.111 sec

* A Docker lite image is used as a base.

Let’s sum it up. Running your Java project in a native image is obviously a more attractive choice in performance and speed. However, you cannot always guarantee it will work correctly because of limitations or work at all, for that matter. Don’t forget that Graal VM with optimizations and other beneficial features is not distributed for free. And the so-called “thin jar layer” container image deployment is only possible when we have jars, i.e. only with a regular runtime.

What we cannot do

Native image executes Java programs in a manner that is different from JVM. It distinguishes between image build time (finding all methods reachable from the entry point) and run time (ahead-of-time compiling these methods without new code). Since the optimization model varies and reducing memory footprint requires a closed-world assumption, you may notice Java applications behaving differently. Also, Native Image does not operate with the original bytecode.

Not all programs can be optimized this way. Limitations that would request a JDK for execution are divided into three big groups:

  1. Class metadata features. Require to be configured at image build time, otherwise lead to a fallback image.
    • Dynamic Class Loading. A class accessed by name at image run time must be included in a configuration file to avoid ClassNotFoundException.
    • Reflection. Accessed elements must be known ahead-of-time. Elements not discovered by static analysis should be configured manually or registered using RuntimeReflection.
    • Dynamic Proxy. java.lang.reflect.Proxy proxies are supported with the closed-world assumptions with all interfaces listed in the configuration file.
    • JCA (Java Cryptography Architecture). The JCA framework relies on reflection for algorithm independence, needs to be configured on its own in Native Image. The security services must be enabled with an option.
    • JNI (Java Native Interface). Java artifacts accessed from JNI should be specified in the configuration.
  2. Features incompatible with closed-world optimization. Not supported and lead to a fallback image immediately.
    • invokedynamic Bytecode and Method Handles. It can introduce calls at run time or change the method that is invoked.
    • Serialization. It could potentially be supported using configuration at image build time unless it was not a constant source of security vulnerabilities.
    • Security Manager. It is proposed to split an application into separate processes to isolate trusted and less trusted code.
  3. Features that may operate differently in Native Image. JVM users are not used to the way they are executed.
    • Signal Handlers. A default setting for image build time is that none are registered unless the user does it explicitly by adding by build option.
    • Class Initializers. Classes are initialized at image run time for compatibility, although it may be limiting. But they optionally can be initialized at build time for faster startup though it may break specific assumptions in existing code.
    • Finalizers. They are not invoked.
    • Threads. Long-deprecated methods in java.lang.Thread are not implemented.
    • Unsafe Memory Access. Fields accessed using sun.misc.Unsafe must be marked.
    • Debugging and Monitoring. Native debuggers and monitoring tools (like GDB or VTune) instead of JVMTI and other Java-targeted tools.

I only have so much space here to discuss native image limitations to running Java apps. For the full detailed overview, head over to GraalVM’s Compatibility and Optimization Guide. And one more thing - static compilation, which requires a lot of time and memory.

Who won?

As is usually the case, it’s a draw. Picking either runtime or native image for building the microservice architecture depends on the tasks at hand, business environment, the state of your organization, and many many other aspects.

So, why does the title have “versus” written in quotes? Because there is no real competition here, no choice between one or the other: they do work in tandem.

The power of “micro-frameworks” for microservices is that they bypass native image restrictions by design (one example is ignoring bytecode instrumentation). The price is lacking capabilities and having limitations imposed on the internal architecture and implementation. Such frameworks as Quarkus, Micronaut, and Helidon follow this route. The goal is to get the almost instant start time and other benefits of native image and GraalVM. Short service start time on regular JVM may be an additional benefit.

Spring Boot, the most widely used Java-based framework, dramatically improved native image support this year; still experimental, it shows impressive performance figures and compatibility levels. Nevertheless, using a certain framework feature, you have to check if it works at all or how it should be configured for the native image. Will it ever be fully transparent? Maybe. Currently, it just goes from another end when the task is to change or extend existing implementation, which relies on reflection and dynamic bytecode.

At the same time, GraalVM is taking steps towards fast and correct work with existing frameworks. Baking of the native image itself is a quite long and resource-consuming process. It involves compilation controlled by a sophisticated configuration that should statically describe all dynamic parts in a system. Automation and speedup are essential for this step. The former is covered by build systems plugins (Maven, Gradle) and tools like Tracing Agent that collect data about microservice needs during a run on a regular JVM. Another step is an actual deployment where devs and ops need to prepare container images, configure and roll them out. So, integration with Kubernetes plays a role when a framework is selected, but it is unlikely to be reflected in the microservice code.

Conclusion

And there you go. This second article in our Microservices Series has hopefully covered all the questions you had regarding JVM and native image. These technologies, while exhibiting different strengths and weaknesses, coexist nicely.

The next and final part will take a wider perspective: we’ll see which place Java takes in the world of microservices besides runtimes and frameworks.

Author image

Dmitry Chuyko

Senior Performance Architect at BellSoft

 Twitter

 LinkedIn

BellSoft LTD [email protected] BellSoft LTD logo Liberica Committed to Freedom 199 Obvodnogo Kanala Emb. 190020 St. Petersburg RU +7 812-336-35-67 BellSoft LTD 199 Obvodnogo Kanala Emb. 190020 St. Petersburg RU +7 812-336-35-67 BellSoft LTD 111 North Market Street, Suite 300 CA 95113 San Jose US +1 702 213-59-59