Introduction
In many client/server applications, there are significant benefits in running multiple instances of the server process. The most important of these benefits are fault tolerance and scalability. However, the success of this approach depends upon the ability to distribute incoming client requests across available server processes, in a manner that is transparent to the client and dynamically able to account for changing server conditions.
When using CORBA as the middleware, no universal solution exists for distributing client requests effectively. The two major ORB (Object Request Broker) vendors, Iona and Inprise, offer the means (products) by which requests can be distributed across multiple instances of an object interface. However, these methods are proprietary and limited in applicability. Therefore, they will not work with other ORBs (see the sidebar). Even so, you can implement a server based on one of these ORBs that performs true load balancing in a way that is both transparent to the client and customizable by the developer. This article describes a VisiBroker-based implementation that provides actual transparent load balancing.
Note that the term load balancing is often used synonymously with request distribution, which is unfortunate since there is an important distinction. Request distribution simply attempts to spread the work among all participating servers without regard to the server's current state or to any other external conditions. Typically, this is accomplished via a round-robin or random selection mechanism. Such techniques are appropriate and sufficient in cases where workload is light, processing times for each request are fairly constant, and server environments are nearly identical. However, if any of these conditions do not hold, these mechanisms can break down quickly, with some servers getting backed up while others are starved. These cases require a more sophisticated approach. Load balancing attempts to remedy this situation by adding intelligence to the distribution mechanism and taking into account these other factors when selecting the server to process a request.
This article assumes that you have a basic understanding of CORBA concepts, such as IDL, interfaces, and ORBs. If you're new to CORBA, you can find out more about it from the material listed in the CORBA Reading section at the end of this article.
Intercepting a Client Request
Inprise foresaw the need for developers to modify the normal processing of requests and provided hooks, called interceptors, into the request-processing layer of the ORB. Subclassing VISServerInterceptor creates a server-side interceptor. There are also client-side and bind interceptors. This class defines eleven methods that are called by the ORB at various stages while processing a client request.
Some of the methods defined in this class include locate, receive_request, prepare_reply, and send_reply. By overriding these methods, code can be added to perform such actions as logging and encryption and, as is the case with the locate method, can alter the normal flow of processing. Figure 1 shows a typical sequence of events that occur when the client obtains a reference to a server object and invokes a method on that object.
The client ORB sends a locate message to the server when the client performs a _bind or when a generic Object reference is narrowed to its more specific sub-type. As its name implies, the locate message is sent out when a client needs to find a specific object implementation. Normally, the server ORB would check that the object exists within the server and respond with an OBJECT_HERE. However, if an interceptor has been installed, the locate method can redirect or forward the locate message to another destination. If the locate method returns the Interoperable Object Reference (IOR) of an object, the server ORB will send back that IOR to the client ORB with a status of OBJECT_FORWARD, indicating that the client ORB should look in this other place for the object implementation.
Note that the locate operation happens only when the object is initially bound or narrowed. Once the specific object reference is obtained, all actions on that object result in invocation messages being sent to the server (except where rebinding is enabled). However, for purposes of this article, the locate method is key to developing a transparent load-balancing mechanism.
Design Issues
Two main design issues must be resolved before getting into the code. First, when multiple instances of the server process are running, one server is responsible for accepting incoming client requests and forwarding them to the other servers (or itself) for processing. Thus, there must be a method for determining this master server. Second, since the master forwards requests to the other slave servers, it must know which slaves are available, and it must be kept current as to each current load for use when forwarding new requests. Implicitly, servers must have the ability to find one another.
The short answer to the second problem is that all servers register with the master server when they are started. At regular intervals of time, they update the master with load information. The significance is that, a server process can contain multiple instances of a particular object implementation (as would most likely be the case in a multithreaded server). So, an implementation can either register the server or register the individual objects within the server.
Figure 2 illustrates both of these alternatives. If only the server is registered, when the master forwards a request to the server, it must forward the request to one of its (co-located) objects. In my implementation, I chose to register individual objects with the master and allow it to forward requests directly to those objects. This simplifies overall design and requires that an interceptor be installed only in the master server.
To determine which server will be the master, one option is that the first server started will be the master. While sounding straightforward, this option is problematic due to difficulty in determining which server was started first, particularly when the servers are running on different systems. Alternatively, the master could be designated on the command line when the server is started. This, however, introduces the possibility of human error (e.g., no master or too many masters).
In either case, both options are not very robust if the master server should fail or be shut down. A better alternative would be for the servers themselves to determine which will be the master. That eliminates the potential for human error, and, if the master server becomes unavailable, the remaining server could select a new master and resume normal operations with no human intervention required.
The solution I chose involves using the CORBA Naming Service. The master server is the one bound to a well-known name in the Naming Service. When a server starts up, it checks for that name. If it does not find the name, it will bind to that name, thereby becoming the master. If the name already exists, the server can be resolved to get a reference to the master server. This solves some of the problems since I now know which is the master (the one bound to a particular name), and I now have a way to reference the master (by resolving that name).
However, this solution is not as simple as it sounds for a couple of reasons. First, the name in the Naming Service could be old, and it could be bound to a non-existent server. (Binding to a non-existent server might be useful if using the Object Activation Daemon (OAD) to launch the server, but I will not be considering that here.) Second, there is no easy way to synchronize the activities of multiple servers when dealing with the Naming Service.
Thus, potential exists for a race condition in which one server finds that the name does not exist and so binds itself as the master. But, before the server can do the binding, another server might check the Naming Service, also find that the name is not bound, and then try to make itself the master. Figure 3 shows the process I used to solve these problems. More robust solutions are possible, such as using the Locate Service, but this solution works simply and sufficiently for most situations.
Finally, a mechanism is required by which slaves can monitor availability of the master. Since the slaves must periodically update the master with load information, this same mechanism can be used to ping the master to make sure it is still alive. Update and ping intervals will be controlled separately, so they can be tuned to the environment. If a ping should fail, the slave re-initiates the above procedure to register a new master in the Naming Service.
Implementation
The implementation of the load-balancing server is based on a load-balancing class, AWSLoadBalancing. Before I discuss this class, I briefly digress to discuss the AWSServer and AWSNames classes. These classes provide a framework that simplifies the task of creating a CORBA server and of working with the Naming Service. They also offer a convenient place to hide some of the complexities of using the load-balancing code. Following this discussion, I will explain some of the more important sections of code in the AWSLoadBalancing class. This article concludes by showing you how to run the test programs and providing sample output.
Server Framework
When writing a CORBA server, certain administrative work is required that has no bearing on your primary task, that is, implementing the business object interface. These "extra" tasks include processing command-line arguments, initializing the ORB and Basic Object Adaptor (BOA), binding and unbinding object names in the Naming Service, and executing the main event loop. The AWSServer class spares the programmer from these repetitive tasks and provides for a very clean main. Figure 4 shows the main of an example server application.
The AWSServer class makes it quick and easy to get a new CORBA server up and running, while hiding details of the particular ORB being used. Implemented as a singleton, it is initialized by the first call to the static instance method that accepts an optional reference to an AWSLoadBalancing object. Specifying this optional argument enables load balancing for this server and changes the object scope in the BOA to local.
The reason for changing the scope is that global CORBA objects are registered automatically with the osagent [1] upon creation, and all global objects can be accessed through a _bind [2], thus circumventing the load-balancing scheme. This is a minor detail since everything will still work even if someone does _bind to the objects. But, it may cause the load-balancing algorithm to be less effective.
Naming Service Utility
The Naming Service is a directory service in which an object reference can be bound to a name. However, dealing with the Naming Service can be very tedious since you have to work with the CosNaming::NamingContext and CosNaming::Name structures when referring to directory elements.
The AWSNames utility allows developers to refer to objects and contexts using simple strings with a more familiar Unix directory-like syntax. For example, "/MyFolder/MyObject" refers to the object MyObject, located in the MyFolder context. The AWSNames utility also lets you use absolute and relative names and contains methods for performing the most common Naming Service operations. These operations include creating and destroying contexts, binding and unbinding names, resolving names, and listing contents of a naming context.
Load Balancing
Before getting into code for the load balancing, one point is especially important. Specifically, the goal is to be able to run multiple identical copies of a server process and have them cooperate with one another to share the workload. Therefore, the load-balancing object must contain the functionality of both a master and a slave and know when to use which.
The IDL [3] for the load-balancing object, shown in Figure 5, defines two interfaces: IProxy and IAgent. The IProxy interface, a marker interface [4], contains no methods or attributes. An object in the master server implements IProxy, and the Naming Service binds it to a name known by the clients. When a client resolves that name and narrows the object reference, the master server eventually receives a locate request for the IProxy interface. The master, in turn, forwards the request to one of the actual business objects. The proxy object is necessary since, as the interceptor code shows, the master server needs it to distinguish between a request from a client for a "real" object and one originating from another server wanting to talk with the load-balancing object.
The class AWSLoadBalancing implements the IAgent interface, which defines methods used by servers to coordinate their activities. It contains methods for registering and unregistering servers and their objects. All methods in the IAgent interface, except for setAgentList, are called only by the slaves on the master.
To update the master with current load information, each slave uses the updateLoadFactor method. The slave passes the load as a double value computed by the virtual function computeLoad. The default implementation returns a fixed value except when compiled on HPUX, where it uses the pstat function to obtain an average load for the host. By overriding the computeLoad method, an application-specific measure of load can be used.
The AWSLoadBalancing class contains two other methods of interest, used only by the master server: pickObject and selectObject. The interceptor calls the pickObject method (shown in Figure 6) when it receives a locate request for an IProxy interface. It returns a reference to one of the available business objects or a null if none could be found.
Although the pickObject method is just a wrapper around the selectObject method, it serves two important purposes. First, it obtains the mutex that prevents other threads from making changes to internal data structures while an object is being selected. Second, it checks to ensure that the object returned by selectObject still exists, thereby removing these responsibilities from selectObject.
The selectObject method contains the logic for selecting the object to service the client request. selectObject is declared as virtual so that custom selection algorithms can be implemented. The default implementation, shown in Figure 7, first selects the server with the minimum load. Then, within that server, it selects objects in a round-robin fashion.
The final detail involves looking at the interceptor that calls the pickObject method. The locate method, defined in VISServerInterceptor, is called when a client needs to find an implementation for a particular interface. The load-balancing interceptor overrides the locate method to handle requests for the IProxy interface (see Figure 8). When a locate request is received, the interface name is extracted. If it is an IProxy, locate responds with the IOR for the object returned by pickObject. Otherwise, locate returns a null value, in which case the ORB will proceed with its normal dispatching activities.
Running the Sample Applications
The source, which accompanies this article, contains all of the code discussed above along with two sample applications, client and server, that demonstrate how it is used (see p. 3 for downloading instructions). To build and run these applications on Windows, you must have Visual C++ and VisiBroker 3.2 or 3.3 with the Naming Service installed. Then complete the following steps:
1. Modify the variable VBROKERDIR defined in the file stdmk to point to the directory where VisiBroker is installed.
2. Build the code using Nmake with the provided makefile. To build the code on another platform or using a different compiler, make the appropriate changes in stdmk.
3. Make sure the VisiBroker agent (osagent) is running.
4. Start the Naming Service. Type:
NameExtF MyRoot myroot.log
5. Start two instances of the server application. Type:
server -SVCnameroot MyRoot
6. Run the client application. Type:
client -SVCnameroot MyRoot /MyFolder/MyObject </p>
Each time you run the client, it tries to resolve in the Naming Service the object name specified on the command line (i.e. /MyFolder/MyObject). If successful, the client invokes the getName method on the object and prints out the results. The getName method returns the name associated with the object, and it will also print the name to the console so that you can tell which server is being accessed. The servers (when compiled with the DEBUG flag defined) send random load factors to the master so that the client should get objects from different servers if run often enough.
Play with the servers: kill the master, start a third, a fourth, a fifth. Run the client several times after each change to make sure that requests are still being routed and that the servers are still responding.
Conclusion
The two major ORB vendors provide basic capabilities for distributing requests to multiple objects. However, these mechanisms fall short of true load balancing, in which a load factor is used when selecting the object to process a request. These mechanisms may be sufficient in situations where all the machines are lightly loaded and of the same capacity, and when request processing times are short and of approximately the same duration.
However, when these conditions are not true or more control is desired, VisiBroker interceptors provide the means for implementing a turnkey solution. In this article, I have shown how such a custom solution can be developed. The AWSLoadBalancing class, which I have presented, can be effectively used as given, or it can be used as a starting point for building your own solution.
Notes
[1] In VisiBroker, the osagent (or Smart Agent) is a dynamic, distributed directory service, which keeps track of the objects available on the network, and is used by the _bind method to locate objects for client applications.
[2] A _bind is a proprietary method, which uses the Smart Agent to locate and return a handle to the appropriate server object.
[3] IDL stands for Interface Definition Language. IDL is a platform- and compiler-neutral language used to describe object interfaces.
[4] A marker is an entity that conveys information by virtue of its being of a certain type. In this implementation, if the interface is of type IProxy, the load-balancing server knows that the request originated with a client, not another server.
Corba Reading
Robert Resendes. "Introduction to CORBA Distributed Objects," C/C++ Users Journal, April 1998.
Seán Baker. CORBA Distributed Objects (Addison-Wesley, 1997). ISBN 0-201-92475-7 (Hardback).
Alan Pope. The CORBA Reference Guide — Understanding the Common Object Request Broker Architecture (Addison-Wesley, 1998). ISBN 0-201-63386-8.
Object Management Group. The Common Object Request Broker: Architecture and Specifications, revision 2.1, Document 97-09-01 (Object Management Group, Framingham, MA, 1997). www.omg.org.
Marc J. Anderson is a Systems Architect in the Austin office of Houston-based SIG, Inc., one of the fastest growing information technology firms, with branches in four U.S. cities. A 14-year IT professional, Marc holds an M.S., Operations Research, from the University of Texas at Austin and a B.S., Math and Computer Science, from the University of Puget Sound. His key capabilities include management of software development teams, web-based design and development, and applied experience in OO methodologies. His areas of expertise are Java and e-commerce, and his primary interests are EJB and CORBA. He can be reached at [email protected].
