From the book
From the book
From the book
1.5 Using Code Generation
In my discussion so far, I process the DSL to populate the Semantic Model (159) and then execute the Semantic Model to provide the behavior that I want from the controller. This approach is what’s known in language circles as interpretation. When we interpret some text, we parse it and immediately produce the result that we want from the program. (Interpret is a tricky word in software circles, since it carries all sorts of connotations; however, I’ll use it strictly to mean this form of immediate execution.)
In the language world, the alternative to interpretation is compilation. With compilation, we parse some program text and produce an intermediate output, which is then separately processed to provide the behavior we desire. In the context of DSLs, the compilation approach is usually referred to as code generation.
It’s a bit hard to express this distinction using the state machine example, so let’s use another little example. Imagine I have some kind of eligibility rules for people, perhaps to qualify for insurance. One rule might be age between 21 and 40. This rule can be a DSL which we can process in order to test the eligibility of some candidate like me.
With interpretation, the eligibility processor parses the rules and loads up the semantic model while it executes, perhaps at startup. When it tests a candidate, it runs the semantic model against the candidate to get a result.
In the case of compilation, the parser would load the semantic model as part of the build process for the eligibility processor. During the build, the DSL processor would produce some code that would be compiled, packaged up, and incorporated into the eligibility processor, perhaps as some kind of shared library. This intermediate code would then be run to evaluate a candidate.
Our example state machine used interpretation: We parsed the configuration code at runtime and populated the semantic model. But we could generate some code instead, which would avoid having the parser and model code in the toaster.
)
Figure 1.5 An interpreter parses the text and produces its result in a single process.
)
Figure 1.6 A compiler parses the text and produces some intermediate code which is then packaged into another process for execution.
Code generation is often awkward in that it often pushes you to do an extra compilation step. To build your program, you have to first compile the state framework and the parser, then run the parser to generate the source code for Miss Grant’s controller, then compile that generated code. This makes your build process much more complicated.
However, an advantage of code generation is that there’s no particular reason to generate code in the same programming language that you used for the parser. In this case, you can avoid the second compilation step by generating code for a dynamic language such as Javascript or JRuby.
Code generation is also useful when you want to use DSLs with a language platform that doesn’t have the tools for DSL support. If we had to run our security system on some older toasters that only understood compiled C, we could do this by having a code generator that uses a populated Semantic Model as input and produces C code that can then be compiled to run on the older toaster. I’ve come across recent projects that generate code for MathCAD, SQL, and COBOL.
Many writings on DSLs focus on code generation, even to the point of making code generation the primary aim of the exercise. As a result, you can find articles and books extolling the virtues of code generation. In my view, however, code generation is merely an implementation mechanism, one that isn’t actually needed in most cases. Certainly there are plenty of times when you must use code generation, but there are even plenty of times where you don’t need it.
Using code generation is one case where many people don’t use a Semantic Model, but parse the input text and directly produce the generated code. Although this is a common way of working with code-generating DSLs, it isn’t one I recommend for any but the very simplest cases. Using a Semantic Model allows you to separate the parsing, the execution semantics, and the code generation. This separation makes the whole exercise much simpler. It also allows you to change your mind; for example, you can change your DSL from an internal to an external DSL without altering the code generation routines. Similarly, you can easily generate multiple outputs without complicating the parser. You can also use both an interpreted model and code generation off the same Semantic Model.
As a result, for most of my book, I’m going to assume that a Semantic Model is present and is the center of the DSL effort.
I usually see two styles of using code generation. One is to generate “first-pass” code, which is expected to be used as a template but is then modified by hand. The second is to ensure that generated code is never touched by hand, perhaps except for some tracing during debugging. I almost always prefer the latter because this allows code to be regenerated freely. This is particularly true with DSLs, since we want the DSL to be the primary representation of the logic that the DSL defines. This means we must be able to change the DSL easily whenever we want to change behavior. Consequently, we must ensure that any generated code isn’t hand-edited, although it can call, and be called by, handwritten code.