Eclipse: Adapting and Updating an IDE

C/C++ Users Journal, September 2003; http://www.ddj.com/documents/s=9710/cuj0309kroening/), I described how and why we updated our Amzi! Prolog proprietary IDE to one based on Eclipse 2. A lot has happened since then, the least of which is that Eclipse is now at Version 3.1. In this article, I describe the move to Eclipse 3, and share user feedback about the new IDE. To recap:

Prolog differs from conventional programming languages in that it is non-procedural and logic-based. It has built-in search and pattern-matching capabilities--so programs run forwards looking for a match, then backwards on failure or to find additional matches. Unlike other Prolog implementations, Amzi! (http://www.amzi.com/) specializes in embedding Prolog logic-bases in conventional (procedural) languages and tools such as Java, C++, .NET, Delphi, and Web servers.

The existing Amzi! IDE dates from the early 1990s and was showing its age. Consequently, a couple of years ago we started thinking seriously about a replacement. We wanted an IDE that would run on multiple platforms, and we wanted to support all the modern conveniences.

The question was how to do that without a budget of the likes of Microsoft or Borland? When we learned of Eclipse, we were immediately intrigued as it offered:

• True open source licensing where we could develop and own our Prolog-specific additions yet contribute to the base product for all to benefit.
  
• Ready-to-run downloads for a wide variety of Windows and UNIX platforms.
  
• Full source code and remote debugging support. The latter is especially important to us because our customers develop Prolog components as part of larger applications instead of stand-alone Prolog programs. Debugging embedded components (especially those running on Web servers) is especially challenging without remote debugging.
  
• Our users could develop both parts of their application using the same IDE. That is, they could develop the user interface in Java or C++ and an intelligent logic-base and/or rule-base in Prolog without ever leaving Eclipse.
  
• The ability to provide international versions and support other languages (half our customers are outside the United States).
• An array of modern conveniences such as projects, syntax coloring, file outlining, Ant build tools, slick source code repository interfaces and automatic file versioning.
  
• The Eclipse Consortium is supported by all the major computer software developers (except Microsoft).
  
• An active developer community providing a variety of extensions for other languages (Java, C++, Cobol) and features (debugging servlets, modeling, even telnet).

However, there were also some downsides:

• Eclipse is written in Java. So the minimum download for the Eclipse Platform (runtime) and Java is over 30 MB. Our current product download is just over 5 MB, so that is a very large increase for our international customers.
  
• The dynamic nature of Eclipse and its extensions presents some performance issues.
  
• Project handling is confusing and inflexible, but improving.

On balance, an Eclipse-based IDE would clearly offer many more features than we could ever hope to build from scratch. But could we get Eclipse to adapt to a Prolog developer's needs for building components in a non-procedural programming language, and at what cost initially and for each major upgrade of Eclipse?

A Pilot Project

To answer these questions, we started with a pilot project to put together a Prolog editor with syntax coloring, an outliner, a project builder and a Prolog interpreter (a.k.a listener). It took a little over a month to get a very rough prototype, but that was enough to convince us that Eclipse was the right choice and that it was flexible enough to handle non-procedural languages.

What to Build?

Before any extensions can be built, one or more plug-ins needs to be created. A plug-in is a set of extensions to Eclipse. To build an entire IDE, there are many extensions organized into multiple plug-ins. A group of related plug-ins that is installed and work together is called a "feature". Typically the user view of a feature is called a "perspective", which is an arrangement of tools, menus and 'views' that are tailored for a specific development task. A view is a window that provides its own menu and buttons and performs a specific task (for example, an editor, outline, task list).

We built an Eclipse feature and perspective for Prolog, and we created extensions to the built-in Debugger perspective for debugging Prolog components. Our extensions are:

• Prolog Source File Editor
  Editor Preferences
  
• Prolog Source File Outliner
  
• Project Cross Referencer
  Project Creator and Properties
  Project Builder (for compiling and linking Prolog modules)
  
• Prolog Listener (for interpreted code)
  Prolog Runtime (for object code)
  Source Code Debugger
  Remote Source Code Debugger
  About Box
  On-Line Help
  Install Site for the Prolog Featuure

You can see the extensions marked with an asterisk in the Prolog perspective in Figure 1.

It took about six months of full-time effort to develop our Prolog perspective and debugger extensions for Prolog code under Version 2 of Eclipse. We added some major editing features in an additional month.

Amzi! Plug-In Architecture

Plug-ins must be in a strict hierarchy, so our user interface plug-in can call our debug plug-in, but not vice versa. Our structure is:

             Prolog User Interface
             <==          |
       Prolog Debugger    |
                <==     <==
                Prolog Core
                    <==
          Amzi! Prolog Virtual Machine

The Prolog Core plug-in provides Java access to the Amzi! Prolog Virtual Machine that is implemented as a dynamic load library (written in C++). Amzi! Prolog is implemented with a virtual machine like Java and .NET. This virtual machine is the basis of all of the Prolog views because we use Prolog to both parse and run Prolog components. The core also provides Prolog predicate names (for editor keyword coloring) and various utility functions.

The Prolog Debugger plug-in contains all the extensions for the Eclipse debugger perspective for both source code and remote debugging. The Prolog User Interface plug-in provides everything else.

This architecture is the only arrangement that worked for us because program launching (the interpreter, debugger or runtime) is part of the user interface and must be able to call the debugger.

Prolog Editor + Outliner + Cross Referencer

The standard Eclipse TextEditor is top-notch and includes keyword coloring, delimiter detection, hover help, completion processing and auto-indent among its many features. Using these features in our Prolog editor was a matter of implementing the Eclipse interfaces for each, and was pretty straightforward.

One feature that was more difficult was breakpoint setting/clearing. Careful experimentation was needed to grasp the subtle interactions between the various editor functions and the code that provide the implementations.

Associated with the editors is the outliner. Our source file outline view provides a tree of the Prolog predicates (see Figure 1). For us, the parser was very easy to build because Prolog can parse Prolog in only a few lines of code.

Once the outline was complete, the cross reference view was easily built as it is also tree-based. Again, it was easy to adapt our existing command-line cross referencing tool (written in Prolog) to be callable from Eclipse (see Figure 2).

Prolog Projects, Properties, and Builder

The primary challenge with projects is how to organize the structure Eclipse provides. Eclipse allows only one project in a directory, which is a new limitation for us that required reorganizing our samples. The questions we had to address were:

* Where will the executable files reside? We decided on an optional bin directory.
* Where will we store the additional project properties? We added a build.properties file that is a format (file extension) that can be viewed/edited by Eclipse.
* What files will be included in the build (compile/link) and loaded when the Prolog Interpreter or Runtime is started? We decided to include only source files in the top-level directory of the project, and files can be excluded in the Project Properties.
* Where will the Amzi! Prolog configuration file reside? We copy one into each project.

Prolog Listener and Runtime

Eclipse provides support for running external tools and re-directing the input/output to a Console View. So we could have simply called our command-line Prolog interpreter. But then we could not take advantage of an IDE and have start/stop buttons, copy/paste and command-line editing.

So to build an Eclipse view for our interpreter (see Figure 3), we needed a calling interface that redirects the user input and output. Getting that to work under Eclipse, though, was another story. Eclipse is very picky about what threads can access the user interface (understandably so). This meant that although our interpreter could run in another thread (so as to not lock up the rest of Eclipse) it has what we could only term as a nasty set of callbacks and synchronized buffers.

The other major challenge for the interpreter and the runtime is Eclipse's launch mechanism. Our architecture caused us problems because Eclipse makes the assumption that for launch, a Java Process will be created to run an executable file.

Since our Prolog is embeddable within Java (and other languages), many of our users never create an operating system executable file that can be run in this way. Instead we call our Prolog virtual machine directly from Java. This led to a number of days struggling with threads. The difficulty is that while Java Processes can be killed, Java Threads cannot. So reliably notifying a thread that it is time to end was most difficult in an environment with input/output being redirected.

Source Code and Remote Debugger

Source code debuggers are one of the most difficult plug-ins to build. The documentation and samples are sparse at best, and the Java, C++, and COBOL debuggers are terribly complex.

The biggest problem we faced with the debugger, and Eclipse, is our desire to highlight the currently executing line of code with either a different icon (not possible in Eclipse 2) or a different color (see Figure 4). We needed to do this because Prolog can be in one of four different states on each line of code (call, fail, redo and exit). We received some excellent help from another developer as well as the Eclipse developers directly (on the mailing lists). Although we did get it to work, we unfortunately have to open a new editor window (in Eclipse 2).

As a side note, we considered attempting a change to the Eclipse source code and submitting it to the developers for possible inclusion in a future release. But then existing Eclipse users could not install our Prolog extensions until that new release was available, so we decided against this.

Out of the six months of development, about half were used for the debugger. However, the good news is once we built the source level debugger, getting it to work remotely was a piece of cake as Eclipse is designed perfectly for this.

User Reactions

Our product is used both for software development and for teaching. Some of the teachers were disappointed with the change because all they needed was a simple user interface (and a small download). But, all of the programmers are delighted with the new tools. The source code debugger has changed how Amzi! Prolog code is developed, and it has improved our own productivity in building new products and maintaining and enhancing Amzi!. User's familiar with some of our competitor's products have commented they didn't have anything like the Eclipse IDE we offer.

Eclipse Version 3

Eclipse 3 introduces both major architectural changes and enhancements to the system. Due to the extent of these changes, the Eclipse developers made great efforts to maintain the old interfaces, while adding the new ones. In theory, this would mean that plug-ins built for Version 2 would run under Version 3. In our case, this did not work. The debugger feature for coloring the type of execution that is occurring on a line of code depended heavily on the Eclipse 2 calling sequences and architecture. Also our launch mechanisms for the Prolog listener performed some unorthodox steps.

So we needed a full conversion to the new architecture. The deprecated interfaces were a big help and let us perform the conversion a step at a time, instead of essentially starting over from the foundation up. It took about a month of development and testing to complete this work (again about half was spent on the debugger). Also, to our delight, the Eclipse team added support for decorating the currently executing line, and we were able to further improve on the color coding and added our own icons (and get rid of the extra editor window).

The rest of our interface remained essentially unchanged, but our users could now take advantage of the new Eclipse features.

Conclusions

We (and our users) can hardly wait for each new release of Eclipse. It keeps getting better and easier to use. It has given us a world-class IDE with all the bells and whistles for a total of about eight months of development work over the course of two years.

Mary Kroening is one of the founders of Amzi! inc. and is the principal author of the Amzi! Eclipse IDE. She can be contacted at [email protected].