Multilevel models in model-driven engineering, product lines, and metaprogramming.

INTRODUCTION

Over the last decade, there has been an increasing desire in both research and practice to abandon the manual development of programs in favor of more automation. (1) Work on software product lines is an example. (2,3) A product line is a family of similar programs. Individual programs differ by the features that they support, where a feature is an increment in program functionality. By modularizing features, programs in a product line are produced by composing features (4); that is, the process of developing a complex program can be reduced to the comparatively simple activities of feature selection and composition. Software tools automate the composition process.

More broadly, research on product lines and generative, transformational, and component-based programming (5) are progressing toward the goal of making programming a computation. This requires a fundamental shift in perspective on program design and development. Programs themselves become objects, and operations on programs are methods of such objects. Metaprogramming is the concept that programming is a computation.

Model-driven engineering (HIDE) is an emerging approach to software development that centers on higher-level specifications of programs in domain-specific languages (DSLs), greater degrees of automation in software development, and the increased use of standards. (6) Among the tenets of MDE is the use of models to represent a program. A model is a specification written in a DSL that captures particular details of a program. As an individual model represents a limited range of information, a program specification is often defined by several models. A model can be computed from other models, and the process of building programs is one of transforming high-level models into executables (considered as yet other models).

Although MDE and metaprogramming are not identical, they do share concepts and goals; namely, that programs are first class (i.e., as objects or models), operations can be performed on them (i.e., as methods or transformations), program development can be a computation, and programs have multiple representations.

In this paper, it is argued that product lines enable both MDE and metaprogramming to converge on a multilevel paradigm of program design. This paradigm not only uses object oriented (OO) design techniques to represent programs that manipulate everyday objects (e.g., employees, books, ledger sheets), but also uses OO techniques to represent the metaprograms that produced these programs, and the meta-metaprograms that produced these metaprograms, recursively. The paradigm is based on a small number of simple and well-known ideas, scales to the synthesis of applications of substantial size, and helps clarify concepts of MDE.

MULTILEVEL DESIGNS

Long before MDE, software engineers realized that a program has many different representations, a simple Java ** application, . class files, and . html files (produced by the javadoc tool). (7) A more elaborate application might have performance models (represented as Mathematica files), makefiles (in Extensible Markup Language [XML] format), formal models (as a state machine in an XML Metadata Interchange [XMI] file), and so on. Each representation is written in a language specific for its purpose--Java is good for source code, HyperText Markup Language (HTML) is good for documentation, and so forth.

Representations can be derived from other representations (e.g., a . class file is derived from its corresponding . java file by javac), or a representation may express unique information about a program (e.g., Mathematica files define a performance model that is not automatically derivable from source files). From this perspective, the software engineering community has been practicing a primitive form of MDE for years.

If we treat files (i.e., representations) as objects that are instances of file types, an OO design emerges. Figure 1 identifies the file types that are encountered in the development of a Java program. File methods are implemented by Java tools that either transform a file into another representation (e.g., .javac is a method that maps a .java file to a .class file) or that modifies the file (e.g., reform is a pretty printing tool that transforms unruly .java files into beautifully formatted files (4)). Even inheritance relationships exist: operating systems provide a standard set of operations (move, copy, delete) on files of all types. Specialized file types are distinguished by different file extensions, and have their own methods (tools).

[FIGURE 1 OMITTED]

A makefile is a program that operates at this level of abstraction. It consists of one or more scripts that create objects (i.e., files) by invoking methods (i.e., tools) in a particular order, and whose goal is to maintain the consistency of these objects whenever an object is modified. (In effect, a makefile is a metaprogram--a program that produces a program.) Makefiles are written in a special language that is not object-oriented, but that could be given a class structure. Figure 2A shows the skeleton of an ant build script, and Figure 2B shows its corresponding class structure (i.e., an ant project is a class, ant tasks are methods, and property definitions are members). Although this correspondence is loose, it is not difficult to recognize an OO class structure in other non-Java artifacts as well. We will return to this observation later in the section entitled "AHEAD."

[FIGURE 2 OMITTED]

Given the above, there are clearly two different levels of abstraction in program design. The meta-application level deals with the construction, manipulation, and synthesis of application level artifacts (files, etc.). Although there are OO languages to express programs at the application level, there is no language that unifies the concepts of files with objects, file instances with file types, tools with methods, and execution scripts (e.g., makefiles) as bodies of methods. (Perhaps Smalltalk and Lisp are exceptions, as they were both programming languages and programming environments.) Not surprisingly, programs at the meta-application level are developed by using tools and design techniques that are reminiscent of those used at the application level in the 1960s. The primary reason that OO techniques now dominate programming is that they impose more structure on programs. More structure means program complexity is better controlled, accidental complexity is reduced, and greater opportunities for automation and analysis arise (e.g., tools that restructure programs using design patterns). On the other hand, programs at the meta-application level are very simple--almost trivially so--compared to their counterparts at the application level. Perhaps this, for no other reason, explains why tools and design techniques at the meta-application level have not progressed.

MDE may be a driving force to change this. MDE explicitly embraces the idea that programs have multiple representations, thus complicating activities at the meta-application level. There are potentially many more program representations, called models, to keep track of; there are many more operations that can be performed on models to modify them or to derive other models. Keeping track of all these representations and relationships itself demands a modeling and programming language. Instead of inventing yet another set of concepts and terminology for this purpose--which itself would be enormously time consuming--why not consider OO as a starting point? The benefits seem clear.

First, OO provides a standard vocabulary and concepts for expressing program designs at any level of abstraction. Enabling researchers and developers to communicate with an established terminology would be extraordinarily beneficial, potentially saving the MDE community years of work. (8,9)

Second, OO provides a reasonable way to think about programs. It formats our thinking so that abstractions are more clearly distinguishable from their possible implementations. For example, today there is an increased interest in graph transformations to manipulate MDE models that are encoded as graphs. (10) While this is fine, let us not forget that there are many program representations that are graphs whose operations are not implemented by graph transformations. A Java program is clearly a graph of classes, yet no javac compiler that I know of is implemented by graph transformations.

These points are elaborated in the following sections by briefly describing projects whose authors unknowingly have used multilevel models to blend model-driven, metaprogramming, and product-line development. The purpose of this paper is to make explicit the notion and value of multilevel models and how to express them.

Let us begin by focusing on product lines whose programs have only one representation, namely source code. This requires the use of an elementary idea, called mixins, which was my first introduction to the idea of programs as objects and operations that map such objects. Once the power of mixins is appreciated, changing to the synthesis of programs with multiple representations is easier to understand.

MIXINS: FUNCTIONS THAT MAP CLASSES

A mixin is a class whose superclass is specified by a parameter. An elementary mixin (where I take liberties on syntax) that adds a color attribute with set and get methods to its input class is: class addColor extends base {

int color = 0;

int getColor( ) { return color; }

void setColor(int c) { color = c; } }