A TCP/IP Socket Location Server

A TCP/IP Socket Location Server

Richard Smereka

You can write remarkably portable network-aware code, if you are careful how you isolate the dependencies.

Introduction

The TCP/IP protocol is one of the most common ways to create network-aware applications. Given the current popularity of TCP/IP as a network protocol and the fact that TCP/IP is used exclusively with Internet applications, this protocol is well worth examining.

The socket resides on top of the TCP/IP protocol. A socket is an IPC (Interprocess Communication) method that allows network applications to communicate with each other. On Unix systems, sockets are implemented using Berkeley sockets. On Windows machines, sockets are constructed using WinSock. This article deals with both types of sockets by presenting a solution for dynamically configuring socket-based client/server applications.

A TCP/IP socket client application that needs to connect to a socket server requires the host name of the socket server, as well as the TCP/IP port number the socket server is listening to. A configuration file or command-line with parameters typically supplies this host name and port number information to clients. This method is manageable only when there are a small number of client and server socket applications within the environment. As the number of socket applications rises, the management of this configuration information becomes more difficult. In addition, changes to the configuration information become more difficult because the host names and TCP/IP port numbers are hard coded. A more dynamic solution is required.

Socloc stands for "socket locate." This facility stores TCP/IP configuration information consisting of host names, TCP/IP port numbers, and, optionally, TCP/IP addresses, in a globally accessible network location. Once this information is stored, socket client applications are not required to know these details. Socloc consists of two major components: a TCP/IP connection-oriented iterative server and an API. All components are written using ANSI C with emphasis on making the code as portable as possible.

When socloc is used effectively by socket servers and socket client applications within an environment, socket servers can easily be moved from one machine to another without any coding or configuration changes to the socket client application. The socloc server also allows multiple instances of the same socket server to run on different computers by using the concept of a service name. Multiple socloc servers may also be used. All socloc servers within an environment synchronize their TCP/IP configuration information. The ability to have multiple socloc and socket servers (of the same type) running provides a good measure of redundancy.

The socloc API also provides for automatic failover of socloc servers. If a socloc server fails or the machine running the socloc server fails, the client application using the socloc API automatically selects and connects to the next available socloc server. It is also very easy to build socket client applications that provide failover protection of a specific socket service as long as more than one available socket server is supplying the required service.

The Socloc Project

Socloc is a large project with many components. Only code fragments are listed here. The complete source code for all socloc components is available via ftp (at www.cuj.com/code).

All components can be compiled and executed on any platform where an ANSI C compiler exists along with the Berkeley socket (under Unix) or WinSock (Windows) interface. At run time, the components require an operational TC/IP network interface. To date, I have compiled and executed the socloc components on many flavors of Unix, as well as 32-bit Windows.

The socloc server and API also make use of a file input/output API, a string parsing API, a file logging API, and a generic socket API. The socket API consists of two modules, a high-level module and a low-level module. The high-level module, ipcsrv.c, contains the server socket structures (shown in Listing 1). Notice that the appropriate socket data structure is selected using OS definitions. These OS definitions are contained in the header file stdhead.h, a portion of which is shown in Listing 2. The specific OS selection specified in stdhead.h automatically determines the path separator, switch character, and platform string.

The Socloc Server

The socloc server (not to be confused with a socket server) is a TCP/IP connection-oriented iterative server. By connection oriented, I mean that the server forms a TCP/IP connection before attempting to receive and send data. I chose this method (TCP, or Terminal Control Protocol) instead of the connectionless-oriented server (UDP, or Universal Datagram Protocol) because the delivery of data is guaranteed with TCP. By iterative, I mean that the socloc server blocks while waiting to receive a connection and processes one request at a time. I chose this method for its simplicity and because of the minimal time required to satisfy a socloc request.

Initializing the Server

Listing 3 shows a portion of the server module socloc.c. This module contains the module global variable WSAData, which is used only in Windows implementations. Any Windows application that uses the socket API must call the WinSock-specific functions WSAStartup and WSACleanup. Listing 3 also contains a two other module global variables: server_port, which contains the current TCP/IP port number serviced by the socloc server, and s_log_file, which is the current log file in use.

The socloc server uses two singly linked lists to manage TCP/IP information. The linked list data structure sloc_entry (shown in Listing 3) holds all socket server TCP/IP information including the service name, machine host name, the port number being serviced, and an optional TCP/IP address. Note that all linked list nodes and string member data are dynamically allocated. The module global sloc_base is a pointer to the base or root node of the sloc_entry linked list. The second linked list, referred to as the configuration list, stores TCP/IP information pertaining to all currently running socloc servers. This configuration list is actually part of the socloc API, which is discussed later. The socloc server uses the socloc API to manage the configuration list and to communicate with other socloc servers.

When the socloc server starts up, the function srv_initialize is called first (shown in Listing 3). This function reads the socloc startup file (described below) and loads the configuration list with this information. srv_initialize then obtains the current host name by calling the TCP/IP function gethostname. This host name is then compared to each entry in the configuration list, looking for a host name match. If srv_initialize finds a match, it uses the associated port number given in the socloc startup file as the TCP/IP port for this server. srv_initialize then marks the matched configuration entry as the local server by calling the socloc API function sl_config_set_ils.

The initialize function then attempts to locate any other currently running socloc servers by calling the socloc API function sloc_find_first_active. Since it is not mandatory that other socloc servers are present, it is acceptable if this attempt to find other socloc servers fails. If other socloc servers are present, the initialization function calls the socloc API function sloc_bcast_config_add, which will register or add the current socloc server to the configuration list of all other known socloc servers. Calling the socloc API function sloc_config_get_list obtains the configuration list of the first responding socloc server. This configuration list is then used to update the current socloc server configuration list.

Processing Requests

Once the socloc server has been registered with other socloc servers and the configuration list is current, main calls the socket API function ipc_init (Listing 3). ipc_init initializes the TCP/IP server socket. After socket initialization is complete, main calls the socket API function ipc_server_wait, which will cause the server to be blocked until a connection is established. Once a client connection is detected, main calls the function process_request to dispatch the client request/command.

The socloc server sends and receives data by using a command-string format called command words. The format consists of a send command code followed by zero or more space-delimited parameters. socloc.h (Listing 4) shows all the socloc send and receive codes. For example, here is a request to the socloc server to add/register a socket server:

60 foo_service thehostname 10000 192.200.45.10

This message sends an add/register request (send code 60) for a socket server with the service name foo_service and host name thehostname on TCP/IP port 10000. The optional TCP/IP address is also included. Once the socloc server has processed this request, it sends a reply to the client application in the same command-word format.

Any send or reply parameter that contains spaces is quoted using single quotes. Technically, any linked list string member can have data that contains spaces including the service name and host name. Practically, it is not a good idea to have spaces in the linked list string data.

The socloc server communicates with all socket servers on a limited basis. When a client application sends a find request, the socloc server first locates the socket server in the linked list and then checks to make sure that the socket server is responding. If the socket server is does not respond, the entry is deleted from the socket server linked list, and the deletion is then communicated to all other known socloc servers. Since the socloc server needs to communicate with all other socket servers, there are a small number of common send and reply codes. These common codes are located in the header file comcode.h (not shown), which is included at the top of Listing 4. Notice that the reply code SL_OK is defined as SOCKET_RC_OK and the send code SL_SEND_STATUS is defined as SOCKET_SEND_STATUS.

There are many features built into the socloc server. The socloc server can add or delete socket server entries, find a socket or socloc server by any criteria, get a copy of the current socket or socloc list, turn the server log on or off, and get the service name and version of the socket or socloc server.

More on Message Formats

When a socloc server responds to a request to return a list of socket or socloc servers, the list will be delimited by the SL_LIST_DELIM character defined in Listing 4. For example:

0 's1 host1 20000^s1 host2 20001^s2 host3 20002'

This reply string starts with the return code followed by the server list, with each list entry having the service name followed by the host name, TCP/IP port number, and an optional TCP/IP address. Since the above reply string contains the service name for each entry, this list is a socket server list. A socloc server list does not contain any service name.

When any find request is sent to the socloc server, the command string has the following syntax:

[service sname] [host hname] [port pnum] [ip ipad]

This command string is called a connect string. Each parameter is actually a parameter pair that starts with a keyword that describes the following parameter value. The service keyword is not allowed when finding socloc servers. The connect string format allows any number of lookup criteria to be used. The only requirement is that there must be at least one parameter pair present. For example:

64 service foobar

When received, this command string requests that the socloc server find (send code 64) a socket server with the service name foobar. Since this command code returns a single socket server entry, the socloc server locates the first entry in the linked list with the same service name. The socloc server then sends a status send code to the socket server. If the socket server responds as expected, the socloc server replies to the client that sent the find request with the matching socket server's TCP/IP information. If, on the other hand, the send command string is:

66 service foobar

The socloc server would return a list of all socket servers providing the service foobar. Unlike the case when it searches for a single server, the socloc server does not perform a status check of all matching servers when it returns a list of servers.

You may have as many socloc servers running within an environment as you like. There are, however, two requirements: no more than one socloc server per machine can be running, and every TCP/IP port number used by each socloc server must be unique [1].

The Socloc API

The socloc API is implemented by four modules. These modules implement the main API, a configuration API, an error message translation API, and an API for assembling and disassembling connect strings. Any application that uses socloc must link with these four modules.

The main API module contains all required client socket structure variables (shown in Listing 5). Before an application calls any socloc API function, it should call the sloc_initialize function (shown in Listing 5). This initialization function first reads the socloc startup file (described below) and loads the configuration linked list by calling the configuration API function sl_config_read. The first socloc server entry is then obtained by calling the configuration API function sl_config_get_first. The initialization function then calls the main API function sloc_status to verify the socloc server entry. The first responding socloc server listed in the socloc startup file is used as the chosen socloc server. The host name and TCP/IP port used by the socloc server are loaded into the module global variables contained in the main API module (shown in Listing 5).

If a chosen socloc server does not respond, the main API then deletes the configuration entry from the linked list and chooses another socloc server. The socloc API has the capability of detecting up to 25 dead socloc servers per request. Once another socloc server has been located, the deleted configuration entries are communicated to all other known socloc servers. This is how the socloc API implements failover of socloc servers. As long as there is another socloc server available, the application using the API continues to run normally.

Building a Socloc-Aware Socket Server

In order to build a socket server that uses socloc, you must link the socket server with the four socloc API modules mentioned above. In addition, you must provide a socloc startup file (discussed below). The socket server service name, host name, TCP/IP port number, and (optionally) the TCP/IP address must be registered with the socloc server. The socket server must also properly respond to all common socket send codes. The socket server must also unregister itself with the socloc server upon normal termination.

No limit exists on the number of socket servers that can be running the same service. You may even have more than one socket server providing the same service on the same machine. The only requirement is that all TCP/IP port numbers must be unique [1].

As an example of a socket server that interfaces with socloc, the CUJ online source code includes a socket server called dumsocks (shown, in part, in Listing 6). The dumsocks server expects a command-line parameter consisting of the TCP/IP port number to be used. All command-line parameters are parsed using the function parse_comline. Once the command-line parameters have been verified, the socket server calls the initialization function srv_initialize. This function initializes the WinSock data structure if the platform is Windows. The socket API function ipc_init then initializes the TCP/IP layer. The socloc API function sloc_initialize then starts the socloc interface. The final step consists of registering the dumsocks server with socloc using the socloc API function sloc_add. After the server has been completely initialized, the server's main function enters an infinite loop that consists of waiting for a connection though the socket API function ipc_server_wait. Once a connection has been established, the process_request function reads the request data, processes the request, and responds accordingly. When the request has been satisfied, the function ipc_close_client closes the client socket.

Building a Socloc-Aware Client

In order for a client application to use socloc, all the socket servers that the client will connect to must be registered with the socloc server. The client application must initialize the socloc API, provide a socloc startup file (discussed below), and locate the required socket server by its service name. In addition, the client application should obtain a current copy of the configuration list from the socloc server and update its own configuration list with this copy. The client should also detect socket communication errors and, if necessary, failover to another socket server with the same service name (if one is available).

An example socket client application, dumsockc, is also given with the online source code. dumsockc interfaces with socloc and the dumsocks server (discussed above). Listing 7 contains part of the application source. The client is initialized in the function d_initialize, which starts the socloc interface and attempts to locate the first responding dumsocks server using the socloc API function sloc_find. The function sloc_config_get_list obtains the current socloc configuration list from the socloc server. The obtained list is then used with the function sl_config_put_list to update the current client socloc configuration list. The dumsockc client then enters a loop that presents a prompt to the user and waits for a command to be typed. The dumsockc client recognizes the user commands shown in Figure 1.

Command keywords may be in upper, lower, or mixed case.

The Socloc Startup File

Every application that uses socloc, including socloc servers, socket servers, and socket clients, requires the presence of a socloc startup file. The startup file is a simple ASCII file that contains a list of suggested socloc servers. Here is a sample startup file:

* comments start with an asterisk as the first non-blank char
* file format is:
*
* hostname port [ip]
*
node1.domain.com 7928
node2.domain.com 7929
node3.domain.com 7930

The startup file is used differently depending on the type of application. The socloc server uses the startup file to locate a line in the startup file where the host name matches the local host name. Once this line is found, the socloc server uses the TCP/IP port number listed on this line. A fatal error occurs if this line is not found. A socket server and socket client application use the startup file as a suggested list of socloc servers. The list does not have to be completely accurate. The only requirement is that at least one of the socloc servers contained in the startup file must respond.

The startup file is read sequentially, and the configuration linked list nodes are added in the same order. One socloc server should be designated as the primary server and another as the secondary server. The primary server should be listed in the socloc startup file first, followed by the secondary server.

As currently defined, the socloc startup file, socloc.ipc, must reside in the current directory when the application is executed.

Conclusion

A socket client application typically needs to know the host name and TCP/IP port number of the socket server. The socket server information is normally hard-coded using configuration files or command-line parameters. This method couples the client application to a specific host and TCP/IP port number.

Socloc presents a method that decouples the socket client application from detailed information about the socket server, which creates a tremendous amount of freedom. Socloc also adds failover protection providing there are redundant servers within the environment.

Socloc can be used with any connection-oriented socket server that will accept a command string. The socloc server assumes that each socket server is connection oriented. There is no provision, at the moment, for connectionless-oriented (UDP) socket servers.

All socloc code is licensed using the GNU GPL (General Program License), which allows anyone to acquire a copy of executable programs and source code free of charge. A full copy of the GPL license is included with the socloc code via ftp.

Note

[1] In fact, all port numbers that are registered with socloc (both socloc servers and socket servers) must be unique.

Further Reading

For more information on Unix socket programming, refer to W. Richard Stevens, Unix Network Programming Volume 1, Second Edition (Prentice Hall, 1998).

For more information on WinSock programming, see Bob Quinn and Dave Shute, Windows Sockets Network Programming (Addison Wesley, 1996).

For further information on socloc, see the complete HTML documentation available via ftp or view the documentation online at the Future Lab website, www.future-lab.com.

Richard Smereka helps run Future Lab Inc., providing system development and programming services specializing in human resource and payroll software and the Open Source movement. You can reach him at [email protected].