Jaison is president of Espial Group and can be contacted at [email protected]. Kumanan is the vice president of research and development for Espial Group and can be contacted at [email protected].
Sidebar: User Interface Dos and Don'ts
Information appliances are network-connected devices that allow for total connectivity whether users are at home, at work, in their cars, or on a plane. Examples of these personal consumer devices include smart web-phones, set-top boxes, personal digital assistants, hand-held computers, and game consoles.
Java was initially developed to be used in consumer devices such as these. However, the language's features also made it a feasible alternative for desktop- and browser-based development. As a result, Java quickly overshot the boundaries of feasibility for the consumer device space.
Realizing that one Java API was not ideal for all development environments, Sun Microsystems released four APIs targeting specific device categories:
- Java for the desktop.
- PersonalJava for consumer devices with diverse displays.
- Embedded Java for nonvisual consumer devices.
- JavaCard for smart cards and similar devices.
In this two-part article, we'll focus on PersonalJava. In this first installment, we'll share some of the lessons we learned about PersonalJava when developing Kalos Espresso, a lightweight Java user interface (UI) toolkit optimized for low-resource environments. In the next installment, we'll put PersonalJava and Kalos Espresso to work and develop an application for a typical information appliance -- a smart web phone.
PersonalJava
PersonalJava is the Java Application Environment (JAE) designed specifically for low-resource environments and diverse visual displays. This JAE consists of an API that caters to the requirements of consumer-device application development. PersonalJava provides interfaces that let you handle different input mechanisms in mouseless and nonkeyboard environments. The API for Personal AWT excludes methods and classes not suited for the consumer-device environment, and adds additional functionality specific to the issues related to such devices. Such specialization has enabled the PersonalJava platform to occupy a small footprint on consumer devices.
The Java Virtual Machine (JVM) has been retooled specifically for this platform and optimized for small-footprint environments. The JVM supports ROM loading, allowing the base PersonalJava system to be deployed in ROM. This leaves more RAM for the applications developed on the platform.
Specifications state that a PersonalJava implementation, which includes the JVM and the class libraries, must fit in 2 MB of ROM and execute in 1 to 2 MB of RAM. These requirements are only for the PersonalJava JAE, and applications that run on top of this platform will have their own memory requirements.
Development for the consumer-device platform is significantly different from developing for the desktop. You need to be able to write code that is optimized for minimal resource environments. Software developed for Java should allocate memory only when required, and release memory as quickly as possible since there is only a limited amount available.
For most consumer devices that do not have a mouse or keyboard, input mechanisms may include pens, remote controls, joysticks, touch, and even speech. The display types and screen sizes for consumer devices also vary and must be considered. It is possible for a display to be a two-color LCD screen, 4-bit LCD screen, 21-inch color TV, or 16-bit TFT color LCD display. These issues lead to the greatest concern, the end user.
Using a consumer device for access to the Internet has created a set of new users who until now did not have an affordable way to access the Internet. Such diverse end-user demographics and the various human interfaces have made human factors an essential part of the development process for consumer-device applications.
When developing applications for consumer devices, you need to be aware of the available resources. This is essential because you need to develop software for limited memory, different displays, and various input mechanisms. A current problem with Java is the inconsistencies in behavior of the various VM implementations. For this reason, applications look different across platforms. To maintain stability and gain control over UI discrepancies, we built Kalos Espresso, a toolkit that's 100 percent pure Java and PersonalJava compliant.
While developing applications for the consumer-device space, we came across several problems related to limited memory, small screen sizes, and human interface issues.
Conserving Memory
One of the big problems with developing software for any small footprint device -- regardless of platform, language, or application -- is conserving memory. PersonalJava presents its own challenges in this area.
Peer versus lightweight components. Development of a visual PersonalJava application requires that the platform implementation of PersonalJava have support of AWT. Currently, some embedded operating-system vendors provide full AWT support, while others provide a minimal implementation of AWT (referred to as Tiny AWT). Most implementations of AWT bind the Java UI components into their respective peer components. This can add a significant amount of overhead, because a peer instance and an AWT instance of the same object need to be maintained. Furthermore, any method calls to the components go through an additional layer of indirection to the peer component.
An alternative approach is to use lightweight components, which do not depend on the system peer resources and are thereby more optimal for consumer device environments. Many vendors sell lightweight components toolkits, but with a footprint of more than 400 KB, they are not feasible for use in small-footprint devices. The size of the toolkit was an important issue when we developed Kalos Espresso as a set of lightweight components that could be used in a limited resource environment. Since we required only a few components to develop a majority of the applications, it is only necessary to include a subset of Kalos Espresso, which is usually around 100 KB.
Resource management. A class loader tool, JavaCodeCompact, works with the PersonalJava virtual machine to load classes into ROM. Since a fair bit of the required classes are loaded into ROM, sharing data between classes can optimize memory usage. JavaCodeCompact maintains a shared string table, which allows for the sharing of similar string occurrences. This shared string table is also sophisticated enough to be able to share substrings of current table elements. A shared table for the various virtual machine data structures is also maintained in order to improve memory usage and performance. Kalos Espresso takes this one step further and implements Optimal Image Resource Management (OIRM).
Figure 1 shows the major parts of an application which supports OIRM, as well as the advantages gained by using the model. The central object in OIRM is ImagePool, a shared image table. This ensures that similar images are never loaded more than once. Instead, an attempt to load a similar image returns a handle to the current element in the shared image table. Another core object is ImageBundle, which is used to wrap an Image before caching it. This object provides additional utility methods that allow you to create altered versions of the image while maintaining only one copy of subsequent images in memory. The unifying principle here is that, since images can take up a lot of space, it is best to avoid keeping more than one copy of a particular image in memory.
This model is best applied to images that see widespread use in an application. It is best not to cache an image that is to be used once and then not used again for a long time. It is better, instead, to directly procure a copy, use it, and dispose of it when it is no longer of any use.
Memory allocation. Java supports garbage collection, which periodically deallocates memory that is not referenced. It is important, however, to allocate memory only when needed and to release memory as quickly as possible. Garbage collection can be assisted by explicitly setting your variables to null as soon as you are done with an object. Also, immediately dispose of images and other heavy graphics objects that are no longer of use when done with them.
User interface optimizations. Labels and graphics are visual cues for users, and help guide them around the UI. Components should support different types of data. Images usually enhance the usability of applications by communicating with the user in a consistent graphical language. In Kalos Espresso, we introduce an Item interface that is the only content supported by each component. An Item is an ownerless, self-drawing, stateless component. An Item has a name, knows how to draw itself, knows its preferred size, can optionally know how to draw itself when selected, and can be cloned. You can create a shared pool of Items that are used by more than one component, and thus conserve memory. An Item is restricted to being entirely visual and contains no additional functionality overhead. Components that use Items as content include lists, tables, and trees. Listing One is an implementation of a SimpleIconItem item.
UIs developed for consumer devices should be able to scale across various screen sizes. To guarantee that this is always the case, you should use layout managers to build UIs. With the proper use and nesting of layout managers, it's possible to achieve any UI layout. The concern with using and nesting too many containers is that more objects are created and more memory is used. While it is possible to use GridBagLayout to create nearly any kind of UI, it can be complex. To facilitate a simpler UI building process, we created specialized layout managers that reduce the number of objects required to build a particular user interface. Listings Two and Three demonstrate a specialized ScrollBarLayout and TableauLayout, which we created to ease Java UI development. Figure 2 shows the results.
What the User Sees
What users see on consumer devices will depend, of course, on the features of the device itself. Among the things you need to keep in mind there are:
Colors. Display types can vary considerably across consumer devices. The screen may be a two-color LCD, four-color LCD, or a 4- or 16-bit color LCD panel. These color restrictions must be taken into consideration while developing applications. It is possible to design usable interfaces for various screen types. Figure 3, for instance, is a sample portion of a UI developed for a two-color, four-color gray, and full-color screen. As you can see, the main difference is between the two-color and four-color interfaces, while the full color is similar to the four-color gray. (For more tips, see the text box entitled "User Interface Dos and Don'ts.")
Visual feedback. Consumer devices are used by a wide variety of audiences, from the technically sophisticated to the novice. If users try to do something and there is no visual feedback, it is possible they will think the device does not work. Static visual feedback might suffer from the same problem, making it extremely important to provide continuous and dynamic feedback. In Kalos Espresso, there are two ways to provide such feedback. You can either use the ProgressBar that shows an incremental counter as a particular operation advances, or Animator (see Listing Four) to show a graphical animation, which keeps the user's attention and provides appropriate visual feedback. It is a good idea to limit such feedback animations to three or four frames so as to conserve resources.
User interface flexibility. Layout managers are useful in building UIs that scale across various screen sizes. However, this method of building UIs provides limited flexibility in positioning components within the user interface. To improve manageability and freely manipulate the UI, we developed advanced borders that provide the flexibility for pixel-level manipulation of components without sacrificing the benefits of scalable layouts. They let users set decorative borders for containers and control background painting. They also allow you to specify an opacity object (background filler), an image, or color. Borders in Kalos Espresso support all of the above and introduce the notion of inner and outer insets with an implicit (yet invisible) boundary between the two. These insets are more visible when using borders and opacity objects, for the perimeter on the inside of the border is controlled with the inner insets, while the outside is controlled using the outer insets. Borders can help you achieve several effects to group components, simplify your UI, and improve usability. Using borders can trim down the number of component objects that would otherwise be required to layout a particular UI. Listing Five shows how you can use borders.
Optimal painting. Since AWT does not support double buffering to eliminate the flicker that occurs when drawing directly to the screen, Kalos Espresso provides a special paint-batching mechanism that takes care of the flicker via the DoubleBufferPanel component. Some devices already support double buffering at the system level, in which case the component should not be used. PersonalJava has an API (see Listing Six) that lets you query the system to find out whether or not double buffering is supported.
Double buffering does use up local memory, but without it, screen flickering can be quite irritating. The trick is to reduce the number of such buffers to the absolute minimum. It is most memory efficient to use only one double buffer per instance of a Frame, Applet, Dialog, or Window. Consumer devices UIs will usually have only one such instance, therefore no more than one DoubleBufferPanel (see Listing Six) should be used.
Another issue to consider when painting components is that you should only repaint the part of the UI that has changed. Kalos Espresso handles this internally by optimizing all repaint method calls. However, you must take care of this for additional extensions and custom components.
User Interaction
There are several ways users can interact with consumer devices -- on-screen keyboards, pens, joysticks, remote controls, specialized buttons on the devices, touch, or even by voice. The nice thing about programming in Java is that you need not be aware of the input mechanism. In cases where more than one input is available, the PersonalJava API provides interfaces that let you set the preferred input for each component. The preferred input interfaces are:
- NoInputPreferred, which can be implemented for components that are usually not selectable (labels, images, and so on).
- KeyboardInputPreferred, which can be implemented for components capable of accepting and processing keyboard and virtual on-screen keyboard inputs.
- ActionInputPreferred, which may be implemented for components that respond to a click or an action from a particular input device.
- PositionalInputPreferred, which may be implemented for components, which will respond to the selection of x-/y-coordinates within the component.
Most applications are designed to furnish the needs of keyboard and mouse users, and applications for the consumer-device arena seem to use the same interfaces. The challenge is to develop a relationship between the UI and input device. Interaction mechanisms have to be designed with care and facilitate the maximum degree of user productivity.
A simple example of designing to this particular space can be seen in some small on-screen keyboards, which have their letters organized in the traditional QWERTY fashion. A question here is, when you have a small keyboard that can only accept input from the touch of a pen or finger, would it not be more useful to have a keyboard with letters arranged in another fashion, alphabetically, for instance?
Next Month
In the next installment of this article, we'll implement some of the techniques we've discussed here in the development of an application for a web phone, an application that's typical of the emerging consumer-oriented class of information appliance.
For More Information
Espial Group Inc.
1200 Elgin St., Suite 205
Ottawa, ON
Canada K2P OP7
613-230-4770
http://www.espial-group.com
DDJ
Listing One
public class SimpleIconItem extends TextItem { private Image m_img;
private int m_gap;
/** Construct an ImageItem.
* @param imgb An image bundle.
* @param name A name for the icon.
* @param gap The space b/w the text and image.
*/
public SimpleIconItem (Image img, String name, int gap) {
super (name);
m_img = img;
m_gap = gap;
}
/* Simple cloning. */
public Object clone () {
try {
SimpleIconItem t = (SimpleIconItem) super.clone ();
t.m_img = m_img;
t.m_gap = m_gap;
return t;
}
catch (Exception x) {
return null;
}
}
/* This method should return true if the item knows to draw differently
* when selected.
* @return Always true */
public boolean canDrawSelected () {
return true;
}
/* This method can be used to change the image associate with the icon.
* @param img An image. */
public void setImage (Image img) {
m_img = img;
}
/* This method can be used to retrieve image associated with the icon.
* @return An instance of Image, or null. */
public Image getImage () {
return m_img;
}
/* This method can be used to change the gap b/w image and text
* @param gap An integer */
public void setGap (int gap) {
m_gap = gap;
}
/* This method can be used to retrieve the gap value. */
public int getGap () {
return m_gap;
}
/* This method returns the optimal dimensions of the item. */
public Dimension getPreferredSize (Component owner, boolean sel) {
Dimension d = super.getPreferredSize (owner, sel);
if (m_img != null) {
int imgw = m_img.getWidth (owner);
int imgh = m_img.getHeight (owner);
int gap = (getName() == null) ? 0 : m_gap;
d.width += imgw + gap + 4;
d.height = FixedMath.max (imgh, d.height) + 4;
}
return d;
}
/* This method draws the image item. */
public void drawItem (Graphics g, Component owner, Rectangle
box, boolean sel) {
int bx = box.x;
int by = box.y;
int bw = box.width;
int bh = box.height;
// position and draw image if any
if (m_img != null) {
int imw = m_img.getWidth (owner);
int imh = m_img.getHeight (owner);
g.drawImage (m_img, bx, ((bh - imh) / 2)+by, owner);
bx += imw + m_gap;
bw -= imw + m_gap;
}
if (getName() != null) {
// now position and draw the text w/o using Item
FontMetrics fm = g.getFontMetrics ();
String str = getName ();
// now draw the text
int txh = fm.getAscent () + fm.getDescent ();
int txy = ((bh - txh) / 2) + by;
if (sel) {
g.setColor (ESystem.FOCUSED_BLUE);
int txw = FixedMath.min (fm.stringWidth (str), bw-2);
g.fillRect (bx-1, txy-1, txw+2+2, txh+2);
g.setColor (Color.white);
}
g.drawString (str, bx+2, txy+fm.getAscent());
}
}
}
</p>
Listing Two
import espial.awt.*;import espial.awt.layout.*;
import espial.awt.borders.*;
import java.awt.*;
</p>
public class ScrollbarLayoutDemo extends Frame {
ScrollbarLayoutDemo () {
super ("ScrollbarLayout Demo");
setBackground (Color.gray);
buildUI ();
}
public Dimension getPreferredSize () {
return new Dimension (340, 450);
}
void buildUI () {
BaseBorder bo;
EContainer p = new EContainer ();
p.setBorder (bo = new BaseBorder ());
bo.setOuterInsets (25, 25, 25, 25);
p.setLayout (new ScrollbarLayout (5, 5));
// the buttons are laid out at the 9 possible positions
p.add (new EButton (" NW "), ScrollbarLayout.NORTHWEST);
p.add (new EButton (" NE "), ScrollbarLayout.NORTHEAST);
p.add (new EButton (" SW "), ScrollbarLayout.SOUTHWEST);
p.add (new EButton (" SE "), ScrollbarLayout.SOUTHEAST);
p.add (new EButton (" North "), ScrollbarLayout.NORTH);
p.add (new EButton (" South "), ScrollbarLayout.SOUTH);
p.add (new EButton (" West "), ScrollbarLayout.WEST);
p.add (new EButton (" East "), ScrollbarLayout.EAST);
p.add (new EButton (" Center "), ScrollbarLayout.CENTER);
setLayout (new GridLayout (1, 1));
add (p);
}
public static void main (String [] args) {
ScrollbarLayoutDemo f = new ScrollbarLayoutDemo ();
f.pack ();
f.show ();
}
}
</p>
Listing Three
import espial.awt.*;import espial.awt.layout.*;
import espial.awt.borders.*;
import espial.awt.item.*;
import java.awt.*;
</p>
class TableauLayoutDemo extends Frame {
ETextField [] m_edInner;
ETextField [] m_edOuter;
EComboBox m_edImageKey;
EComboBox m_edColor;
ECheckbox m_edUseImage, m_edUseColor;
TableauLayoutDemo () {
super ("TableauLayout Demo");
m_edInner = new ETextField [4];
m_edOuter = new ETextField [4];
setBackground (Color.lightGray);
buildUI ();
}
public Dimension getPreferredSize () {
return new Dimension (324, 240);
}
void buildUI () {
EContainer p1, p2, p3;
GroupBorder gbo;
BaseBorder bo;
int i;
// the first tableau layout
p1 = new EContainer ();
p1.setLayout (new TableauLayout (5, 5, TableauLayout.LEFT,
TableauLayout.TOP));
p1.add (new ELabel ("Top"), new Point (1, 0));
p1.add (new ELabel ("Left"), new Point (2, 0));
p1.add (new ELabel ("Bot."), new Point (3, 0));
p1.add (new ELabel ("Right"), new Point (4, 0));
p1.add (new ELabel ("Inner Insets"), new Point (0, 1));
p1.add (new ELabel ("Outer Insets"), new Point (0, 2));
// inner inset fields
p1.add (m_edInner [0] = new ETextField (4), new Point (1, 1));
p1.add (m_edInner [1] = new ETextField (4), new Point (2, 1));
p1.add (m_edInner [2] = new ETextField (4), new Point (3, 1));
p1.add (m_edInner [3] = new ETextField (4), new Point (4, 1));
// outer inset fields
p1.add (m_edOuter [0] = new ETextField (4), new Point (1, 2));
p1.add (m_edOuter [1] = new ETextField (4), new Point (2, 2));
p1.add (m_edOuter [2] = new ETextField (4), new Point (3, 2));
p1.add (m_edOuter [3] = new ETextField (4), new Point (4, 2));
</p>
// the second tableau layout
p2 = new EContainer ();
p2.setLayout (new TableauLayout (5, 5, TableauLayout.LEFT,
TableauLayout.TOP));
ECheckboxGroup cg = new ECheckboxGroup ();
p2.add (m_edUseColor = new ECheckbox ("Background Color",
false, cg), new Point (0, 0));
m_edUseColor.setCentered (false);
p2.add (m_edColor = new EComboBox (), new Point (1, 0));
m_edColor.add (new TextItem ("No Color"));
m_edColor.select (0);
p2.add (m_edUseImage = new ECheckbox ("Background Image",
false, cg), new Point (0, 1));
m_edUseImage.setCentered (false);
p2.add (m_edImageKey = new EComboBox (), new Point (1, 1));
m_edImageKey.add (new TextItem ("No Image"));
m_edImageKey.select (0);
// a border layout two hold the two tableaus together
EFauxWindowContainer fwc = new EFauxWindowContainer ();
p3 = fwc.getBackFaceContainer ();
p3.setLayout (new BorderLayout (0, 5));
p3.add (p2, BorderLayout.NORTH);
p3.add (p1, BorderLayout.CENTER);
//////////
// and something for the frame.
setLayout (new GridLayout (1, 1));
add (fwc);
}
public static void main (String [] args) {
TableauLayoutDemo f = new TableauLayoutDemo ();
f.pack ();
f.show ();
}
}
</p>
Listing Four
import espial.awt.*;import espial.awt.accessories.*;
import espial.awt.borders.*;
import espial.awt.item.*;
import java.awt.event.*;
import espial.image.*;
</p>
import java.awt.*;
</p>
public class AnimPage extends EContainer implements ActionListener {
private AnimationView anim;
private EButton start, stop;
public AnimPage (Image [] frames) {
buildUI (frames);
}
private void buildUI (Image [] frames) {
anim = new AnimationView (frames, 120);
setLayout (new BorderLayout ());
add (anim, BorderLayout.CENTER);
EContainer p0 = new EContainer ();
p0.setLayout (new FlowLayout (FlowLayout.CENTER, 20, 5));
p0.add (start = new EButton (" Start ", null, EButton.TOP));
start.addActionListener (this);
start.setTip ("Begin the animation");
p0.add (stop = new EButton (" Stop ", null, EButton.TOP));
stop.addActionListener (this);
stop.setEnabled (false);
stop.setTip ("Stop the animation.");
add (p0, BorderLayout.SOUTH);
}
public void actionPerformed (ActionEvent e) {
Object src = e.getSource ();
if (src == start ) {
start.setEnabled (false);
stop.setEnabled (true);
anim.startAnimation ();
start.repaint ();
stop.repaint ();
}
else if (src == stop) {
stop.setEnabled (false);
start.setEnabled (true);
anim.stopAnimation ();
start.repaint ();
stop.repaint ();
anim.repaint ();
}
}
}
</p>
Listing Five
public class BordersExample extends Frame implements WindowListener { /* Protected constructor used internally to set up the application. */
protected BordersExample () {
super ("BordersExample");
addWindowListener (this);
setBackground (Color.lightGray);
buildUI ();
}
/* The method which holds the EXAMPLE code. Study the code to understand
* how to use the class this example demonstrates. */
public void buildUI () {
DoubleBufferPanel dbp = new DoubleBufferPanel ();
/////////// EXAMPLE STUDY CODE
BaseBorder bo;
dbp.setLayout (new GridLayout (3, 3));
EContainer p0 = new EContainer ();
p0.setLayout (new FlowLayout ());
bo = new BaseBorder (Color.white);
bo.setOuterInsets (5, 5, 5, 5);
p0.setBorder (bo);
p0.add (new ELabel ("BaseBorder"));
dbp.add (p0);
p0 = new EContainer ();
p0.setLayout (new FlowLayout ());
bo = new LineBorder (Color.white);
bo.setOuterInsets (5, 5, 5, 5);
p0.setBorder (bo);
p0.add (new ELabel ("LineBorder"));
dbp.add (p0);
</p>
p0 = new EContainer ();
p0.setLayout (new FlowLayout ());
bo = new BevelBorder (true);
bo.setOuterInsets (5, 5, 5, 5);
p0.setBorder (bo);
p0.add (new ELabel ("BevelBorder-Raised"));
dbp.add (p0);
</p>
p0 = new EContainer ();
p0.setLayout (new FlowLayout ());
bo = new BevelBorder (false);
bo.setOuterInsets (5, 5, 5, 5);
p0.setBorder (bo);
p0.add (new ELabel ("BevelBorder-Lowered"));
dbp.add (p0);
</p>
p0 = new EContainer ();
p0.setLayout (new FlowLayout ());
bo = new GroupBorder (new TextItem ("Group Border"));
bo.setOuterInsets (5, 5, 5, 5);
p0.setBorder (bo);
dbp.add (p0);
///////////
setLayout (new BorderLayout ());
add (dbp, BorderLayout.CENTER);
}
/* Application starts out sized at 450x480 pixels. */
public Dimension getPreferredSize () {
return new Dimension (450, 480);
}
/* Invoked when a window has been opened. */
public void windowOpened(WindowEvent e) {
}
/* Invoked when a window is in the process of being closed.
* The close operation can be overridden at this point. */
public void windowClosing(WindowEvent e) {
dispose ();
System.exit (0);
}
/* Invoked when a window has been closed. */
public void windowClosed(WindowEvent e) {
}
/* Invoked when a window is iconified. */
public void windowIconified(WindowEvent e) {
}
/* Invoked when a window is de-iconified. */
public void windowDeiconified(WindowEvent e) {
}
/* Invoked when a window is activated. */
public void windowActivated(WindowEvent e) {
}
/* Invoked when a window is de-activated. */
public void windowDeactivated(WindowEvent e) {
}
/* The method used to start the application by the JVM. */
public static void main (String [] arg) {
BordersExample f = new BordersExample ();
f.pack ();
f.show ();
}
}
</p>
Listing Six
class MyApp extends Frame { ...
// we decide if we want to use double buffering explicitly or rely
// on pjava (if it provides it)
EContainer p = null;
Class comp = Component.class;
try {
Method dblchk = comp.getMethod ("isDoubleBuffered", new Class [0]);
// method exists
Boolean chk = (Boolean) dblchk.invoke (this, null);
if (chk.booleanValue()) {
// no need for double buffering ourselves
p = new EContainer ();
}
}
catch (NoSuchMethodException x) {
}
if (p == null) {
// check if we made one, else force double buffering
p = new DoubleBufferPanel ();
}
...
}
DDJ
Copyright © 1999, Dr. Dobb's Journal
