The Eclipse Modeling Framework

Frank is an engineer in IBM's Software Group and lead author of Eclipse Modeling Framework: A Developer's Guide (Addison-Wesley, August 2003). He can be contacted at [email protected].

The idea of building applications by first modeling them, then transforming these models into implementation code has been around for many years. Providing a higher level abstraction for defining software would seem to be a natural evolution. Twenty or so years ago, structured programming languages replaced assembly language, or machine code, as the most popular way to write software. About 10 years ago, object-oriented programming languages became entrenched as the most predominant languages, again raising the abstraction level. Lately, there's been a lot of talk about model-driven development as the next higher level abstraction.

Each step in the evolution of software development has been accompanied by skepticism, and model-driven development is no different. The skepticism is usually the result of overly grandiose visions and promises, opening up the visionaries to attack from the more practical types. Many programmers think that class diagrams might be helpful to document their designs, but they know that implementing complex systems by simply "transforming a picture" is a pipe dream. They know that all the expressive power of a programming language can't be available in a model because if it was, it wouldn't be any simpler (higher level). It would just be another programming language.

That said, most programmers do recognize that generating some of the code that they write over and over must be possible. How many copy-and-paste operations do you need to do before you start to wonder if there couldn't be a way of specifying parameters for patterns that you want, and just have the code generated automatically? Clearly, these patterns must represent some higher level abstraction that, if only it could be specified (modeled), could enable us to write a lot less code. The problem is that the lack of simple and seamless ways to enter the parameters for these patterns has tended to limit the use of automatic code generation in mainstream development environments.

So, to foster the evolution of widespread model-driven development, a low-cost way to gradually introduce it is first needed. The Eclipse Modeling Framework (EMF; http://www.eclipse.org/emf/) has emerged as a middle-ground in the modeling versus programming worlds. Integrated with the Eclipse Java development tools, EMF provides an easy way for you to define models, from which many common code-generation patterns are generated. It leverages the fact that just about every program we write manipulates some kind of data model. It might be defined using Java, UML, XML Schema, or some other definition language. EMF aims to extract this intrinsic "model," thereby giving programmers an easy way to get some of the benefits of modeling, without necessarily becoming a full-fledged modeler overnight.

Defining the Model

Have you wondered how many of the applications you write manipulate data? The answer is pretty close to 100 percent. So where is the description of the data that your program is manipulating? Is the data simply defined by the interfaces in a Java program? Maybe you have an XML Schema that defines the data? Maybe it's a relational database (RDB) schema? Maybe you're already a modeling enthusiast and have a UML class diagram? Whatever the case may be, you either already have a "data model," or are about to implicitly define it in the code you're going to write.

Say that you want to write a program to manage purchase orders for some store or supplier. A purchase order includes a "bill to" and "ship to" address, and a collection of (purchase) items. An item includes a product name, quantity, and a price. If you're a Java programmer, you'd probably start by writing some Java interfaces, something like Listing One.

The abstraction provided by Java itself doesn't explicitly define anything more than simple interfaces and methods, but through naming convention and comments, these interfaces do define the model. For example, the get/set method pairs (or Bean properties, if you want to look at it that way) represent data, or attributes, in the model. The getItems() method indicates that the purchase order aggregates, or is associated with (references), its corresponding items. Using this model information, one could easily imagine generating an implementation of many of these methods.

Another common way to describe a data model like this one is by using an XML Schema. In this case, the same model information could be provided using XML Schema complexType and element constructs; see Listing Two. Notice that XML Schema is serving two purposes here. In addition to defining the model, it's also specifying the persistent format of the XML file, into which the data will presumably be serialized. This dual-purpose role of XML Schema can lead to complexity, especially when some of the more advanced constructs are used, but it is becoming one of the most popular ways of modeling data structures these days.

A third, and the simplest, way to represent the purchase order model would be with a UML class diagram, as in Figure 1. Not surprisingly, the UML diagram is the most concise representation. As you can see, it provides all the same information as the other two forms of the model, only this time in a simple picture.

Identifying an application's data model has two big advantages:

• It gives you a nice, high-level way to both communicate the design and to share data with other applications.
• You can potentially generate some, if not all, of the implementation code.

Today's applications are becoming increasingly less monolithic and more integrated with other applications. The key to supporting fine-grain data integration between applications is a common understanding of the data—that is, a model—and ideally a common implementation framework. Being able to generate some of the implementation code is also the key to increased productivity and high quality and reliability of these increasingly complex integrated applications.

Modeling with EMF

EMF is a framework and toolkit that extends Eclipse's Java Development Tooling into the world of model-driven development. EMF provides a metamodel (that is, a model of a model), called "Ecore," for describing EMF models. Ecore, which is essentially the class diagram subset of UML, is based on the Object Management Group's (OMG) Meta Object Facility (MOF) specification.

The canonical form of an EMF model is an XML Metadata Interchange (XMI) serialization of an Ecore model. However, one of the main strengths of EMF is its flexibility with respect to the means by which the Ecore model can be defined:

• XMI; you can create an (Ecore) XMI document directly, using an XML or text editor, or using EMF's simple tree-based sample Ecore editor.
• UML; you can define the model using a commercial UML modeling tool such as Rational Rose, or using an Eclipse plug-in such as Omondo's free EclipseUML graphical editor (available at http://www.omondo.com/).
• Java Interfaces; basic Java interfaces with a few simple (@model) annotations can be used to describe an Ecore model.
• XML Schema; an XML Schema defining the data structures for the model, can be converted to an Ecore model.
• Others; EMF is a highly extensible and customizable framework/toolkit. Support for other forms of model definition is not only possible, but expected.

The first approach is the most direct, but generally appeals only to XML gurus. The second choice is the most desirable if you are already using modeling tools, while the Java approach provides the benefits of modeling in a pure Java development environment like Eclipse's JDT. The XML Schema approach is most desirable when the tool or app is intended to manipulate XML data that is already defined using an XML Schema (as with web services). Regardless of which input form is used to define an EMF model, the benefits are the same.

Given an Ecore model of the classes, or data, of the application, EMF provides a surprisingly large portion of the benefits of modeling. The mapping between an EMF (Ecore) model and Java is natural and simple for Java programmers to understand. At the same time, it's enough to support fine-grain data integration between applications and provide a significant productivity gain resulting from code generation.

From an Ecore model, EMF's generator can create a corresponding set of Java implementation classes. Every generated EMF class extends from the framework base class, EObject, which enables the objects to integrate and work in the EMF runtime environment. EObject provides an efficient reflective API for accessing the object's properties generically. In addition, change notification is an intrinsic property of every EObject and an adapter framework can be used to support open-ended extension to the objects. The runtime framework also manages bidirectional reference handshaking, cross-document referencing including demand-load, and arbitrary persistent forms with a default generic XMI serialization that can be used for any EMF model. EMF even provides support for dynamic models; that is, Ecore models that are created (in memory), and then instantiated without generating code. All of the benefits of the EMF runtime framework apply equally to them.

In addition to the core EMF framework, another framework, EMF.Edit, extends and builds on the core, adding support for generating adapter classes that enable viewing and command-based (undoable) editing of a model, and even a basic working model editor.

Generating a Model

It is straightforward to use EMF to convert an XML Schema (the purchase order schema example) into a model, then generate an implementation of the model including an XML model editor for instances of the schema. I start from an Eclipse Workbench with the purchase order schema (SimplePO.xsd) in its workspace; see Figure 2. To create the EMF (Ecore) model from the schema, all you need to do is use the EMF Project Wizard to import the model:

1. Invoke File->New->Project... and select EMF Project.
2. Name the project, that is, "simplepo."
3. Select Load from an XML Schema and then locate the file SimplePO.xsd.
4. Proceed through the remaining wizard pages, following instructions and accepting any default values.

Doing this creates a new Eclipse Java project initially containing two model-related files with the following suffixes:

• ecore, an XMI serialization of the Ecore model corresponding to our XML schema.
• genmodel, a generator model, which is a wrapper (decorator) model of the .ecore model that provides options relevant only to the code generator (for example, the directories in which to generate the code, and choices for other user-selectable implementation patterns).

At this point, you have extracted the EMF model of the application and can now display the UML equivalent of your XML Schema by opening the .ecore file using a graphical UML editor, such as Omondo's EclipseUML. Given that the XML Schema-based model doesn't include graphical layout information, the UML model is presented using a default layout. The purchase order model looks nice, but more complicated models may need a bit of graphical editing.

To generate the model's corresponding Java interfaces, as well as implementation code, close the .ecore editor and open the .genmodel using the EMF code generator. A popup menu in the generator lets you generate model implementation code (Generate Model Code), adapters that support editing and display (Generate Edit Code), and even a working model editor (Generate Editor Code). Generate All generates all three. Both the Ecore and the generator models (.ecore and .genmodel) are EMF models themselves. That is, Ecore is its own metamodel, as well as the metamodel for the generator model. Both models have been generated with the EMF code generator, and the EMF code generator is even an EMF-generated editor, like the one just generated for the purchase order model.

Now that you've generated the Java interfaces and classes, you can look at the third form (Java) of the model by opening the Java files using the Eclipse Java Editor; see Figure 3. Because you're inside the Eclipse Java Development Environment, the generated Java code has already been compiled and is ready to run. Launching a runtime workbench, you can now create and edit XML instance documents to edit purchase orders and their corresponding items. The generated editor, although simple, is surprisingly powerful, even supporting drag-and-drop and unlimited undo. After editing the purchase order, you can save the file by invoking File->Save. The serialized form will, as expected, be an XML instance document conforming to the original schema (in SimplePO.xsd); see Figure 4.

The generated EMF editor provides a good example of the recommended style for EMF model editors and can be used as either a test application or as a starting point from which to start customizing. This is practical because of another important advantage of EMF: You can regenerate a model after changing any of the generated source code, without wiping out your changes. The EMF generator produces files that are intended to be a combination of generated pieces and hand-written pieces. You are expected to edit the generated classes to add methods and instance variables. You can always regenerate from the model as needed, and your additions will be preserved during the regeneration.

Code Generation Patterns

Code generating tools often generate code that isn't particularly clean, simple, or elegant—or even intended to be looked at. It's just meant to be a behind-the-scenes function that you simply rely on to work. EMF's code generation, on the other hand, is expected to be viewed, extended, and even modified, so a great deal of effort went into designing clean and elegant patterns—the kind of code you would write by hand. The intention is to solve the repetitive and sometimes difficult problems that you face over and over. Among the available patterns are:

Feature Access and Notification. Simple EMF features (properties) are generated using the standard JavaBean pattern. For example, the get method for the shipTo attribute of the purchase order example simply returns an instance variable like Listing Three. The corresponding set method simply sets the same variable, but it also sends a notification to any observers that may be interested in the state change; see Listing Four. Notice that to make this method more efficient when the object has no observers, the relatively expensive call to eNotify() is avoided by the eNotificationRequired() guard.

More complicated patterns are generated for other types of features, especially bidirectional references where referential integrity is maintained. In all cases, however, the code is generally as efficient as possible, given the intended semantic.

Bidirectional Reference Handshaking. The implementation pattern generated for bidirectional references provides a great example of the value of EMF's code generation. Look at an example of a simple doubly linked list of purchase orders. In UML, it might look something like Figure 5. Here you define a simple 1-to-1 bidirectional relationship, which could be used to create a chain of purchase orders. The "next" and "previous" references are used to access a purchase order's following and previous order, respectively. By defining this as a bidirectional reference, you're saying that the "next" reference of a given purchase order and the "previous" reference of its following order should always be kept in sync.

If you use Java interfaces to declare this same association, you use @model annotations in the Javadoc to indicate that the two references, "next" and "previous," are opposites (that is, reverses) of each other, rather than simply a pair of independent references; see Listing Five. Because the two references are opposites, you can set the association from either end, and the other end is automatically updated. This may even involve removing references from other objects. For example, consider the case where three purchase orders (p1, p2, and p3) are initially linked as in Figure 6(a). If you now want to change the "next" reference of p1 to be p3, instead of p2, call the setNext() method on p1:

p1.setNext(p3);

As a result of executing this single statement, the three objects in Figure 6(b) are updated.

As you can see, there is significantly more happening than simply updating the "next" reference of p1. In fact, all of this happens:

• Remove p1.next pointer to p2.
• Remove p2.previous pointer to p1.
• Set p1.next pointer to p3.
• Remove p3.previous pointer to p2.
• Remove p2.next pointer to p3.
• Set p3.previous pointer to p1.
• Send notifications (changes to p1, p2, and p3).

To do this, the generated EMF implementation of the getNext() method looks something like Listing Six. Notice that, in addition to adding and removing all of the required references, the notifications are queued and only dispatched at the very end when all of the objects are in their final consistent state.

Although implementing something like this isn't that hard for good programmers, the bottom line is that it isn't trivial and can be hard to get right. By simply leveraging the higher level EMF concept of a bidirectional relationship, we get an efficient, proven, correct implementation pattern for free.

Object Persistence (Proxy Resolution and Demand Load)

The ability to persist, and reference other persisted model objects, is one of the most important benefits of EMF modeling; it's the foundation for fine-grain data integration between applications. The EMF framework provides simple, yet powerful, mechanisms for managing object persistence.

Many EMF models use the default XMI serialization provided by EMF to persist their models. Others implement customized serialization, allowing the model to be persisted in its natural (native) format. For example, XML Schema models can be made persistent as .xsd files, Java models as .class files, and so on. Other models, for example, mappings, are typically persisted using (the default) XMI, although specific types of mappings may be serialized differently. For example, XML Schema mappings can also be serialized in XSLT format.

Regardless of the actual persistence format, the EMF framework provides generic support for resource saving and loading, including cross-resource linking with proxy resolution and demand load. An object serialized in an XMI file, for example, can reference another object serialized in another format, or possibly even stored in a relational database. EMF's resource cross-referencing and demand-loading mechanisms tie them together with one simple and uniform Java API.

Conclusion

EMF is an open-source project at Eclipse.org providing a framework and code-generation facility for building robust applications based on surprisingly simple models. Models can be defined in several different ways—Java interfaces, XML Schemas, UML—from which EMF will generate a large part of the application. The generated code is clean, efficient, and easily hand modified. You can even regenerate the model after changing the code, without wiping out your changes.

EMF provides low-cost modeling for the Java mainstream by leveraging the intrinsic model in an application. With EMF, no high-level modeling tool is required. There are two fundamental benefits from using EMF. First, it results in a productivity gain by providing a nice, high-level way (UML) to communicate the design and then generate part of the implementation code. Second, EMF allows applications to integrate at a much finer granularity than is otherwise possible. The designers of EMF believe that it mixes just the right amount of modeling with programming to maximize the effectiveness of both.

DDJ

Listing One

public interface PurchaseOrder {
  String getShipTo();
  void setShipTo(String value);
  String getBillTo();
  void setBillTo(String value);
  List getItems(); // List of Item
}

public interface Item {
  String getProductName();
  void setProductName(String value);
  int getQuantity();
  void setQuantity(int value);
  float getPrice();
  void setPrice(float value);
}

<xsd:complexType name="PurchaseOrder">
 <xsd:sequence>
  <xsd:element name="shipTo" type="xsd:string"/>
  <xsd:element name="billTo" type="xsd:string"/>
  <xsd:element name="items"  type="PO:Item" 
               minOccurs="0" maxOccurs="unbounded"/>
 </xsd:sequence>
</xsd:complexType>

<xsd:complexType name="Item">
 <xsd:sequence>
  <xsd:element name="productName" type="xsd:string"/>
  <xsd:element name="quantity" type="xsd:int"/>
  <xsd:element name="price" type="xsd:float"/>
 </xsd:sequence>
</xsd:complexType>

public String getShipTo() {
  return shipTo;
}

public void setShipTo(String newShipTo) {
  String oldShipTo = shipTo;
  shipTo = newShipTo;
  if (eNotificationRequired())
    eNotify(new ENotificationImpl(this, Notification.SET,
                       POPackage.PURCHASE_ORDER__SHIP_TO, oldShipTo, shipTo));
}

public interface PurchaseOrder {
  /** @model opposite="previous" */
  PurchaseOrder getNext();

  /** @model opposite="next" */
  PurchaseOrder getPrevious();
  ...
}

public void setNext(PurchaseOrder newNext) {
  if (newNext != next) {
    NotificationChain msgs = null;
    if (next != null)
      msgs = ((InternalEObject)next).eInverseRemove(this,
        POPackage.PURCHASE_ORDER__PREVIOUS, PurchaseOrder.class, msgs);
    if (newNext != null)
      msgs = ((InternalEObject)newNext).eInverseAdd(this,
        POPackage.PURCHASE_ORDER__PREVIOUS, PurchaseOrder.class, msgs);
    msgs = basicSetNext(newNext, msgs);
    if (msgs != null) msgs.dispatch();
  }
  else if (eNotificationRequired())
    eNotify(new ENotificationImpl(this, Notification.SET,
     POPackage.PURCHASE_ORDER__NEXT, newNext, newNext)); // touch notification
}