Compiling and Scripting
This chapter is from the book
This chapter is from the book
This chapter is from the book
This chapter introduces two techniques for processing code. You can use the compiler API when you want to compile Java code inside your application. The scripting API lets you invoke code in a scripting language such as JavaScript or Groovy.
8.1. The Compiler API
There are quite a few tools that need to compile Java code. Obviously, development environments and programs that teach Java programming are among them, as well as testing and build automation tools. Another example is the processing of Jakarta Server Pages—web pages with embedded Java statements.
8.1.1. Invoking the Compiler
It is very easy to invoke the compiler. Here is a sample call:
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler(); OutputStream outStream = . . .; OutputStream errStream = . . .; int result = compiler.run(null, outStream, errStream, "-sourcepath", "src", "Test.java");
A result value of 0 indicates successful compilation.
The compiler sends its output and error messages to the provided streams. You can set these arguments to null, in which case System.out and System.err are used. The first argument of the run method is an input stream. As the compiler takes no console input, you can always leave it as null. (The run method is inherited from the generic javax.tools.Tool interface, which allows for tools that read input.)
The remaining arguments of the run method are the arguments that you would pass to javac if you invoked it on the command line. These can be options or file names.
8.1.2. Launching a Compilation Task
You can have more control over the compilation process with a CompilationTask object. This can be useful if you want to supply source from string, capture class files in memory, or process the error and warning messages.
To obtain a CompilationTask, start with a compiler object as in the preceding section. Then call
JavaCompiler.CompilationTask task = compiler.getTask( errorWriter, // Uses System.err if null fileManager, // Uses the standard file manager if null diagnosticListener, // Uses System.err if null options, // null if no options classes, // For annotation processing; null if none sources);
The last three arguments are Iterable instances. For example, a sequence of options might be specified as
Iterable<String> options = List.of("-d", "bin");
The sources parameter is an Iterable of JavaFileObject instances. If you want to compile disk files, get a StandardJavaFileManager and call its getJavaFileObjects method:
StandardJavaFileManager stdFileManager
= compiler.getStandardFileManager(null, null, null);
Iterable<JavaFileObject> sources
= stdFileManager.getJavaFileObjectsFromStrings(List.of("File1.java", "File2.java"));
JavaCompiler.CompilationTask task
= compiler.getTask(null, null, null, options, null, sources);
The getTask method returns the task object but does not yet start the compilation process. The CompilationTask class extends Callable<Boolean>. You can submit it to an ExecutorService for concurrent execution, or you can just make a synchronous call:
Boolean result = task.call();
8.1.3. Capturing Diagnostics
To listen to error messages, install a DiagnosticListener. The listener receives a Diagnostic object whenever the compiler reports a warning or error message. The DiagnosticCollector class implements this interface. It simply collects all diagnostics so that you can iterate through them after the compilation is complete.
var collector = new DiagnosticCollector<JavaFileObject>();
compiler.getTask(null, fileManager, collector, null, null, sources).call();
for (Diagnostic<? extends JavaFileObject> d : collector.getDiagnostics())
{
System.out.println(d);
}
A Diagnostic object contains information about the problem location (including file name, line number, and column number) as well as a human-readable description.
You can also install a DiagnosticListener to the standard file manager, in case you want to trap messages about missing files:
StandardJavaFileManager stdFileManager = compiler.getStandardFileManager(diagnosticsListener, null, null);
8.1.4. Reading Source Files from Memory
If you generate source code on the fly, you can have it compiled from memory, without having to save files to disk. Use this class to hold the code:
public class StringSource extends SimpleJavaFileObject
{
private String code;
StringSource(String name, String code)
{
super(URI.create("string:///" + name.replace(".", "/") + ".java"), Kind.SOURCE);
this.code = code;
}
public CharSequence getCharContent(boolean ignoreEncodingErrors)
{
return code;
}
}
Then generate the code for your classes and give the compiler a list of StringSource objects:
List<StringSource> sources = List.of( new StringSource(className1, class1CodeString), new StringSource(className2, class2CodeString), . . .); task = compiler.getTask(null, fileManager, diagnosticsListener, null, null, sources);
8.1.5. Writing Byte Codes to Memory
If you compile classes on the fly, there is no need to save the class files to disk. You can save them to memory and load them right away.
First, here is a class for holding the bytes:
public class ByteArrayClass extends SimpleJavaFileObject
{
private ByteArrayOutputStream out;
ByteArrayClass(String name)
{
super(URI.create("bytes:///" + name.replace(".", "/") + ".class"), Kind.CLASS);
}
public byte[] getCode()
{
return out.toByteArray();
}
public OutputStream openOutputStream() throws IOException
{
out = new ByteArrayOutputStream();
return out;
}
}
Next, you need to configure the file manager to use these classes for output:
var classes = new ArrayList<ByteArrayClass>();
StandardJavaFileManager stdFileManager
= compiler.getStandardFileManager(null, null, null);
JavaFileManager fileManager
= new ForwardingJavaFileManager<>(stdFileManager)
{
public JavaFileObject getJavaFileForOutput(Location location,
String className, Kind kind, FileObject sibling)
throws IOException
{
if (kind == Kind.CLASS)
{
ByteArrayClass outfile = new ByteArrayClass(className);
classes.add(outfile);
return outfile;
}
else
return super.getJavaFileForOutput(location, className, kind, sibling);
}
};
To load the classes, you need a class loader (see Chapter 9):
public class ByteArrayClassLoader extends ClassLoader
{
private Iterable<ByteArrayClass> classes;
public ByteArrayClassLoader(Iterable<ByteArrayClass> classes)
{
this.classes = classes;
}
public Class<?> findClass(String name) throws ClassNotFoundException
{
for (ByteArrayClass cl : classes)
{
if (cl.getName().equals("/" + name.replace(".", "/") + ".class"))
{
byte[] bytes = cl.getCode();
return defineClass(name, bytes, 0, bytes.length);
}
}
throw new ClassNotFoundException(name);
}
}
After compilation has finished, call the Class.forName method with that class loader:
ByteArrayClassLoader loader = new ByteArrayClassLoader(classes); Class<?> cl = Class.forName(className, true, loader);
8.1.6. An Example: Dynamic Java Code Generation
In the Jakarta Server Pages (JSP) technology for dynamic web pages, you can mix HTML with snippets of Java code, for example:
<p>The current date and time is <b><%= LocalDateTime.now() %></b>.</p>
The JSP engine dynamically compiles the Java code into a servlet. In our sample application, we use a simpler example and generate code for a very simple worksheet format. Authors include text, Java expressions (enclosed in backticks), and Java statements (enclosed in triple backticks) into a worksheet. The worksheet is compiled into a Java class that prints the text and the values of the expressions, and executes the Java statements. Here is a sample worksheet:
What is your name? ``` var in = new Scanner(System.in); String name = in.nextLine(); ``` How old are you? ``` int age = in.nextInt(); ``` Hello, `name`! Next year, you will be `age + 1`.
This is a very minimal text version of a programming notebook (https://docs.jupyter.org/en/latest/#what-is-a-notebook). The sheet is translated into a class whose main method has print statements to emit the text and expression values. The statements are included verbatim. One can envision an enhancement of this program that displays images and tables, just like a real programming notebook.
The buildSource method in the program of Listing 8.1 builds up this code and places it into a StringSource object. That object is passed to the Java compiler.
As described in the preceding section, we use a ForwardingJavaFileManager that constructs a ByteArrayClass object for every compiled class. These objects capture the class file that is generated when the class is compiled. After compilation, we use the class loader from the preceding section to load the class and execute its main method.
This example demonstrates how you can use dynamic compilation with in-memory source and class files.
Listing 8.1 compiler/CompilerTest.java
1 package compiler;
2
3 import java.io.*;
4 import java.nio.file.*;
5 import java.util.*;
6 import java.util.List;
7
8 import javax.tools.*;
9 import javax.tools.JavaFileObject.*;
10
11 /**
12 * @version 1.2 2023-08-16
13 * @author Cay Horstmann
14 */
15 public class CompilerTest
16 {
17 public static void main(final String[] args)
18 throws IOException, ReflectiveOperationException
19 {
20 JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
21
22 var classFileObjects = new ArrayList<ByteArrayClass>();
23
24 var diagnosticsListener = new DiagnosticCollector<JavaFileObject>();
25
26 JavaFileManager fileManager
27 = compiler.getStandardFileManager(diagnosticsListener, null, null);
28 fileManager = new ForwardingJavaFileManager<>(fileManager)
29 {
30 public JavaFileObject getJavaFileForOutput(Location location,
31 String className, Kind kind, FileObject sibling) throws IOException
32 {
33 if (kind == Kind.CLASS)
34 {
35 var fileObject = new ByteArrayClass(className);
36 classFileObjects.add(fileObject);
37 return fileObject;
38 }
39 else return super.getJavaFileForOutput(location, className, kind, sibling);
40 }
41 };
42
43
44 String mainClassName = "compiler.Main";
45 String worksheetName = args.length > 0 ? args[0] : "compiler/worksheet";
46
47 StandardJavaFileManager stdFileManager
48 = compiler.getStandardFileManager(null, null, null);
49 var sources = new ArrayList<JavaFileObject>();
50 for (JavaFileObject file : stdFileManager.getJavaFileObjectsFromStrings(
51 List.of(mainClassName.replace(".", "/") + ".java")))
52 sources.add(file);
53
54 JavaFileObject source = buildSource(mainClassName, Path.of(worksheetName));
55 JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager,
56 diagnosticsListener, null, null, List.of(source));
57 Boolean result = task.call();
58
59 for (Diagnostic<? extends JavaFileObject> d : diagnosticsListener.getDiagnostics())
60 System.out.println(d.getKind() + ": " + d.getMessage(null));
61 fileManager.close();
62 if (!result)
63 {
64 System.out.println("Compilation failed.");
65 System.exit(1);
66 }
67
68 var loader = new ByteArrayClassLoader(classFileObjects);
69 Class<?> mainClass = loader.loadClass(mainClassName);
70 mainClass.getMethod("main", String[].class).invoke(null, new Object[] { args });
71 }
72
73 /**
74 * Builds the source for the class that implements the worksheet actions.
75 * @param className the name of the class to be built
76 * @param worksheet the file containing the worksheet instructions
77 * @return a file object containing the source in a string builder
78 */
79 static JavaFileObject buildSource(String className, Path worksheet)
80 throws IOException
81 {
82 var builder = new StringBuilder();
83 int n = className.lastIndexOf(".");
84 if (n >= 0)
85 {
86 String packageName = className.substring(0, n);
87 className = className.substring(n + 1);
88 builder.append("package ").append(packageName).append(";\n\n");
89 }
90 builder.append("import java.util.Scanner;");
91 builder.append("public class ").append(className).append(" {\n");
92 builder.append("public static void main(String[] args) {\n");
93 boolean verbatim = false;
94 for (String line : Files.readAllLines(worksheet))
95 {
96 if (line.equals("```")) verbatim = !verbatim;
97 else if (verbatim) builder.append(line).append("\n");
98 else
99 {
100 String[] fragments = line.split("`");
101 for (int i = 0; i < fragments.length; i++)
102 {
103 builder.append("System.out.print(");
104 if (i % 2 == 0)
105 builder.append("\"")
106 .append(fragments[i].replace("\\", "\\\\").replace("\"", "\\\""))
107 .append("\"");
108 else
109 builder.append(fragments[i]);
110 builder.append(");\n");
111 }
112 builder.append("System.out.println()\n;");
113 }
114 }
115
116 builder.append("} }\n");
117 return new StringSource(className, builder.toString());
118 }
119 }