- Introduction
- Understanding the Java Platform Module System
- From Monolithic to Modular: The Evolution of the JDK
- Continuing the Evolution: Modular JDK in JDK 11 and Beyond
- Implementing Modular Services with JDK 17
- JAR Hell Versioning Problem and Jigsaw Layers
- Open Services Gateway Initiative
- Introduction to Jdeps, Jlink, Jdeprscan, and Jmod
- Conclusion
This chapter is from the book
This chapter is from the book
This chapter is from the book
JAR Hell Versioning Problem and Jigsaw Layers
Before diving into the details of the JAR hell versioning problem and Jigsaw layers, I’d like to introduce Nikita Lipski, a fellow JVM engineer and an expert in the field of Java modularity. Nikita has provided valuable insights and a comprehensive write-up on this topic, which we will be discussing in this section. His work will help us better understand the JAR hell versioning problem and how Jigsaw layers can be utilized to address this issue in JDK 11 and JDK 17.
Java’s backward compatibility is one of its key features. This compatibility ensures that when a new version of Java is released, applications built for older versions can run on the new version without any changes to the source code, and often even without recompilation. The same principle applies to third-party libraries—applications can work with updated versions of the libraries without modifications to the source code.
However, this compatibility does not extend to versioning at the source level, and the JPMS does not introduce versioning at this level, either. Instead, versioning is managed at the artifact level, using artifact management systems like Maven or Gradle. These systems handle versioning and dependency management for the libraries and frameworks used in Java projects, ensuring that the correct versions of the dependencies are included in the build process. But what happens when a Java application depends on multiple third-party libraries, which in turn may depend on different versions of another library? This can lead to conflicts and runtime errors if multiple versions of the same library are present on the classpath.
So, although JPMS has certainly improved modularity and code organization in Java, the “JAR hell” problem can still be relevant when dealing with versioning at the artifact level. Let’s look at an example (shown in Figure 3.5) where an application depends on two third-party libraries (Foo and Bar), which in turn depend on different versions of another library (Baz).
)
Figure 3.5 Modularity and Version Conflicts
If both versions of the Baz library are placed on the classpath, it becomes unclear which version of the library will be used at runtime, resulting in unavoidable version conflicts. To address this issue, JPMS prohibits such situations by detecting split packages, which are not allowed in JPMS, in support of its “reliable configuration” goal (Figure 3.6).
)
Figure 3.6 Reliable Configuration with JPMS
While detecting versioning problems early is useful, JPMS does not provide a recommended way to resolve them. One approach to address these problems is to use the latest version of the conflicting library, assuming it is backward compatible. However, this might not always be possible due to introduced incompatibilities.
To address such cases, JPMS offers the ModuleLayer feature, which allows for the installation of a module sub-graph into the module system in an isolated manner. When different versions of the conflicting library are placed into separate layers, both of those versions can be loaded by JPMS. Although there is no direct way to access a module of the child layer from the parent layer, this can be achieved indirectly—by implementing a service provider in the child layer module, which the parent layer module can then use. (See the earlier discussion of “Implementing Modular Services with JDK 17” for more details.)
Working Example: JAR Hell
In this section, a working example is provided to demonstrate the use of module layers in addressing the JAR hell problem in the context of JDK 17 (this strategy is applicable to JDK 11 users as well). This example builds upon Nikita’s explanation and the house service provider implementation we discussed earlier. It demonstrates how you can work with different versions of a library (termed basic and high-quality implementations) within a modular application.
First, let’s take a look at the sample code provided by Java SE 9 documentation:1
1 ModuleFinder finder = ModuleFinder.of(dir1, dir2, dir3);
2 ModuleLayer parent = ModuleLayer.boot();
3 Configuration cf = parent.configuration().resolve(finder, ModuleFinder.of(), Set.of("myapp"));
4 ClassLoader scl = ClassLoader.getSystemClassLoader();
5 ModuleLayer layer = parent.defineModulesWithOneLoader(cf, scl);
In this example:
At line 1, a ModuleFinder is set up to locate modules from specific directories (dir1, dir2, and dir3).
At line 2, the boot layer is established as the parent layer.
At line 3, the boot layer’s configuration is resolved as the parent configuration for the modules found in the directories specified in line 1.
At line 5, a new layer with the resolved configuration is created, using a single class loader with the system class loader as its parent.
)
Figure 3.7 JPMS Example Versions and Layers
)
Figure 3.8 JPMS Example Version Layers Flattened
Now, let’s extend our house service provider implementation. We’ll have basic and high-quality implementations provided in the com.codekaram.provider modules. You can think of the “basic implementation” as version 1 of the house library and the “high-quality implementation” as version 2 of the house library (Figure 3.7).
For each level, we will reach out to both the libraries. So, our combinations would be level 1 + basic implementation provider, level 1 + high-quality implementation provider, level 2 + basic implementation provider, and level 2 + high-quality implementation provider. For simplicity, let’s denote the combinations as house ver1.b, house ver1.hq, house ver2.b, and house ver2.hq, respectively (Figure 3.8).
Implementation Details
Building upon the concepts introduced by Nikita in the previous section, let’s dive into the implementation details and understand how the layers’ structure and program flow work in practice. First, let’s look at the source trees:
ModuleLayer
├── basic
│ └── src
│ └── com.codekaram.provider
│ ├── classes
│ │ ├── com
│ │ │ └── codekaram
│ │ │ └── provider
│ │ │ └── House.java
│ │ └── module-info.java
│ └── tests
├── high-quality
│ └── src
│ └── com.codekaram.provider
│ ├── classes
│ │ ├── com
│ │ │ └── codekaram
│ │ │ └── provider
│ │ │ └── House.java
│ │ └── module-info.java
│ └── tests
└── src
└── com.codekaram.brickhouse
├── classes
│ ├── com
│ │ └── codekaram
│ │ └── brickhouse
│ │ ├── loadLayers.java
│ │ └── spi
│ │ └── BricksProvider.java
│ └── module-info.java
└── tests
Here’s the module file information and the module graph for com.codekaram.provider. Note that these look exactly the same for both the basic and high-quality implementations.
module com.codekaram.provider {
requires com.codekaram.brickhouse;
uses com.codekaram.brickhouse.spi.BricksProvider;
provides com.codekaram.brickhouse.spi.BricksProvider with com.codekaram.provider.House;
}
The module diagram (shown in Figure 3.9) helps visualize the dependencies between modules and the services they provide, which can be useful for understanding the structure of a modular Java application:
The com.codekaram.provider module depends on the com.codekaram.brickhouse module and implicitly depends on the java.base module, which is the foundational module of every Java application. This is indicated by the arrows pointing from com.codekaram.provider to com.codekaram.brickhouse and the assumed arrow to java.base.
The com.codekaram.brickhouse module also implicitly depends on the java.base module, as all Java modules do.
The java.base module does not depend on any other module and is the core module upon which all other modules rely.
The com.codekaram.provider module provides the service com.codekaram.brickhouse.spi.BricksProvider with the implementation com.codekaram.provider.House. This relationship is represented in the graph by a dashed arrow from com.codekaram.provider to com.codekaram.brickhouse.spi.BricksProvider.
)
Figure 3.9 A Working Example with Services and Layers
Before diving into the code for these providers, let’s look at the module file information for the com.codekaram.brickhouse module:
module com.codekaram.brickhouse {
uses com.codekaram.brickhouse.spi.BricksProvider;
exports com.codekaram.brickhouse.spi;
}
The loadLayers class will not only handle forming layers, but also be able to load the service providers for each level. That’s a bit of a simplification, but it helps us to better understand the flow. Now, let’s examine the loadLayers implementation. Here’s the creation of the layers’ code based on the sample code from the “Working Example: JAR Hell” section:
static ModuleLayer getProviderLayer(String getCustomDir) {
ModuleFinder finder = ModuleFinder.of(Paths.get(getCustomDir));
ModuleLayer parent = ModuleLayer.boot();
Configuration cf = parent.configuration().resolve(finder,
ModuleFinder.of(), Set.of("com.codekaram.provider"));
ClassLoader scl = ClassLoader.getSystemClassLoader();
ModuleLayer layer = parent.defineModulesWithOneLoader(cf, scl);
System.out.println("Created a new layer for " + layer);
return layer;
}
If we simply want to create two layers, one for house version basic and another for house version high-quality, all we have to do is call getProviderLayer() (from the main method):
doWork(Stream.of(args)
.map(getCustomDir -> getProviderLayer(getCustomDir)));
If we pass the two directories basic and high-quality as runtime parameters, the getProviderLayer() method will look for com.codekaram.provider in those directories and then create a layer for each. Let’s examine the output (the line numbers have been added for the purpose of clarity and explanation):
1 $ java --module-path mods -m com.codekaram.brickhouse/
com.codekaram.brickhouse.loadLayers basic high-quality
2 Created a new layer for com.codekaram.provider
3 I am the basic provider
4 Created a new layer for com.codekaram.provider
5 I am the high-quality provider
Line 1 is our command-line argument with basic and high-quality as directories that provide the implementation of the BrickProvider service.
Lines 2 and 4 are outputs indicating that com.codekaram.provider was found in both the directories and a new layer was created for each.
Lines 3 and 5 are the output of provider.getName() as implemented in the doWork() code:
private static void doWork(Stream<ModuleLayer> myLayers){
myLayers.flatMap(moduleLayer -> ServiceLoader
.load(moduleLayer, BricksProvider.class)
.stream().map(ServiceLoader.Provider::get))
.forEach(eachSLProvider -> System.out.println("I am the " + eachSLProvider.getName() +
" provider"));}
In doWork(), we first create a service loader for the BricksProvider service and load the provider from the module layer. We then print the return String of the getName() method for that provider. As seen in the output, we have two module layers and we were successful in printing the I am the basic provider and I am the high-quality provider outputs, where basic and high-quality are the return strings of the getName() method.
Now, let’s visualize the workings of the four layers that we discussed earlier. To do so, we’ll create a simple problem statement that builds a quote for basic and high-quality bricks for both levels of the house. First, we add the following code to our main() method:
int[] level = {1,2};
IntStream levels = Arrays.stream(level);
Next, we stream doWork() as follows:
levels.forEach(levelcount -> loadLayers
.doWork(…
We now have four layers similar to those mentioned earlier (house ver1.b, house ver1.hq, house ver2.b, and house ver2.hq). Here’s the updated output:
Created a new layer for com.codekaram.provider My basic 1 level house will need 18000 bricks and those will cost me $6120 Created a new layer for com.codekaram.provider My high-quality 1 level house will need 18000 bricks and those will cost me $9000 Created a new layer for com.codekaram.provider My basic 2 level house will need 36000 bricks and those will cost me $12240 Created a new layer for com.codekaram.provider My high-quality 2 level house will need 36000 bricks and those will be over my budget of $15000
The variation in the last line of the updated output serves as a demonstration of how additional conditions can be applied to service providers. Here, a budget constraint check has been integrated into the high-quality provider’s implementation for a two-level house. You can, of course, customize the output and conditions as per your requirements.
Here’s the updated doWork() method to handle both the level and the provider, along with the relevant code in the main method:
private static void doWork(int level, Stream<ModuleLayer> myLayers){
myLayers.flatMap(moduleLayer -> ServiceLoader
.load(moduleLayer, BricksProvider.class)
.stream().map(ServiceLoader.Provider::get))
.forEach(eachSLProvider -> System.out.println("My " + eachSLProvider.getName()
+ " " + level + " level house will need " + eachSLProvider.getBricksQuote(level)));
}
public static void main(String[] args) {
int[] levels = {1, 2};
IntStream levelStream = Arrays.stream(levels);
levelStream.forEach(levelcount -> doWork(levelcount, Stream.of(args)
.map(getCustomDir -> getProviderLayer(getCustomDir))));
}
Now, we can calculate the number of bricks and their cost for different levels of the house using the basic and high-quality implementations, with a separate module layer being devoted to each implementation. This demonstrates the power and flexibility that module layers provide, by enabling you to dynamically load and unload different implementations of a service without affecting other parts of your application.
Remember to adjust the service providers’ code based on your specific use case and requirements. The example provided here is just a starting point for you to build on and adapt as needed.
In summary, this example illustrates the utility of Java module layers in creating applications that are both adaptable and scalable. By using the concepts of module layers and the Java ServiceLoader, you can create extensible applications, allowing you to adapt those applications to different requirements and conditions without affecting the rest of your codebase.