Window subclassing is probably the most versatile tool in the user-interface programmer’s toolkit. It gives you total freedom to do wonderful things or, as the case may be, to shoot yourself in the foot. This month’s column describes wdjSUB, a general subclassing library that improves your chances of avoiding self-mutilation.
Subclassing
Window subclassing is subversion, pure and simple. When you subclass a window, you replace its window function with your own, and the original gets to see messages only if your function decides to pass them on. Usually you want to be good about this. If you interfere any more than necessary, the whole exercise is pointless. If you greedily keep all messages to yourself, you would have been better off developing a new window class from scratch.
Window subclassing comes in several variants:
Instance subclassing: This technique (also known as “local subclassing”) dynamically replaces the window function of an existing window instance. You directly alter the window procedure by calling SetWindowLong() to alter the GWL_WNDPROC window data. This affects only a single window. Other windows are unaltered, even if they are of the same class.
Global subclassing: This means changing the window function that resides in the window class definition. You use SetClassLong() (with the GCL_WNDPROC parameter) to store your own window function, and any new window of that class will be assigned your window function instead of the one the window class was originally created with. Windows that already exist before you call SetClassLong() are unaffected by this technique.
Class cloning: Also known as superclassing, this is just another way to borrow functionality from an existing window class. You use GetClassInfo() and RegisterClass() to register a new window class that has the same characteristics as an existing window class, but with a new window function.
This article covers the first and most dynamic of these techniques, instance subclassing. The most obvious reason for instance subclassing is to modify the behavior of standard windows such as edit controls. Less obvious, perhaps, is subclassing to listen in on the message traffic: a ToolTip control, for example, will subclass its parent if you set the TTF_SUBCLASS flag. This lets the ToolTip control act on mouse messages sent to the parent, saving said parent from having to send TTM_RELAYEVENT messages to the ToolTip control. The ToolTip control thus becomes a more self-contained widget.
The archetypal subclassing example subclasses a specific edit control in a dialog box to filter out illegal characters. Before the advent of the ES_NUMBER style, this was a popular technique for creating numeric input fields.
Basic Instance Subclassing
To subclass a window, you first need a place to store the address of the original window function:
static WNDPROC orgWndProc;
Next, you need a window function to replace the original; call it myWndProc(). The main difference between myWndProc() and other window functions is that myWndProc() invokes orgWndProc instead of DefWindowProc() to handle messages it doesn’t want to handle itself:
CallWindowProc( orgWndProc, hwnd, msg, wParam, lParam );
It’s imperative that you invoke the original window function through CallWindowProc(). Never call orgWndProc directly; it may not even be a function address! If you subclass a Unicode window with a non-Unicode window function under Windows NT, the system must translate text-related message parameters, and the alleged window function is actually a handle to some black-box data structure cooked up by SetWindowLong().
You use SetWindowLong() to both retrieve the original window procedure and store your new window procedure. For example, if you were creating a dialog box that subclassed one of its edit controls, you might execute this statement when handling WM_INITDIALOG:
orgWndProc = (WNDPROC) SetWindowLong(
GetDlgItem( hwnd, ID_OF_EDIT_CONTROL_TO_BE_SUBCLASSED ),
GWL_WNDPROC, (LONG) myWndProc );
Finally, the original window function must be restored before the window dies. The typical myWndProc() does this in response to either WM_DESTROY or WM_NCDESTROY:
SetWindowLong( hwnd, GWL_WNDPROC, (LONG) orgWndProc );
Subclassing Issues
The biggest problem with the previous code fragments is the static variable orgWndProc that holds the original window function. This works for subclassing a single window or for subclassing multiple windows that share the same window function at subclassing time. It does not work in general because of the difficulty involved in stuffing multiple function pointers into a single variable. Using multiple variables wouldn’t help either, as you’d have no way of telling them apart from the context of myWndProc().
The first and foremost requirement for a general subclassing library, then, is a way of associating saved window functions with HWNDs. Here are some other desirable properties:
• It should be possible to apply multiple subclassings to a window, i.e., to create a subclassing chain.
• It should be possible to subclass and unhook in any order.
• It is often convenient to associate arbitrary subclassing- specific data with a window; a general solution should offer this feature. Consider a toolbar that subclasses its parent to listen for WM_SIZE messages. From the context of the subclassing window function, which receives the parent HWND, the toolbar’s HWND is not immediately available without such a facility.
• The application programming interface (API) should be easy to use and protect against programming mistakes. Runtime robustness is also desirable, of course.
• It may be desirable to apply one subclassing multiple times to the same window. As an example, multiple toolbars might wish to subclass their common parent with the same subclassing to listen for WM_SIZE messages.
wdjSUB satisfies all these requirements except the last. It can be done, but the resulting design is not what you’d call elegant. A better solution permitting repeated subclassing uses a single subclassing to listen for WM_SIZE and maintains its own list of interested windows (which could include status bars, as well as toolbars).
Associating Data with Windows
Data can be associated with windows in two basic ways: you can associate the data with the window itself (using SetWindowLong() or window properties), or you can maintain a static list of associations (array, linked list, binary tree, hash table, Post-it notes, or engraved stone tablets).
Every window has a spare 32-bit word that you can access by passing GWL_USERDATA to SetWindowLong(), so that might appear the ideal place to store your custom data. If, however, if you don’t have the source code to the window you’re subclassing, you have no guarantee that the window doesn’t use GWL_USERDATA for its own purposes.
You could try to work around this by having your subclass procedure dynamically restore the old value of GWL_USERDATA before passing any message on to the previous window procedure, putting your own data back in when that procedure returns. Allocating the following structure would give you room to save everything you need:
struct ORIGINAL {
LONG lOriginalUserData;
WNDPROC fnOriginalWndProc;
};
GWL_USERDATA would then hold a pointer to this structure. This approach has a serious problem, though — it doesn’t stand up to recursion. Consider WM_PAINT: before passing WM_PAINT to the next window function in the subclassing chain, you restore the old GWL_USERDATA to, say, zero. Before CallWindowProc() returns, someone somewhere will call BeginPaint(), which may send a WM_ERASE message to the window. The WM_ERASE message arrives; you attempt to retrieve the original pointer from GWL_USERDATA and get...zero. Oops.
Window Properties
Storing the original window function as a window property works better, the main difference being that you no longer need to restore any original values. The only requisite is that each subclassing use a unique property name. With subclassing chains, this suggests an image of saved window functions sticking out all over the window like pins from a pincushion; I hereby dub this the “porcupine technique.”
Conventional wisdom has it that GetProp() is less efficient than GetWindowLong(), so I did some informal testing to see just how big a problem this might be. In my benchmarking, I found that GetWindowLong() was faster than GetProp() by an order of magnitude when the property name was a string. When the property name was an atom, however, the tables were turned — GetProp() was now slightly faster than GetWindowLong().
Don’t read too much into this benchmarking. It’s only valid for a Windows NT 4.0 window with a single property. It does indicate, though, that the use of window properties shouldn’t cause excessive performance problems.
Practicing Safe Unhooking
Unhooking is just a matter of reverse subclassing, with one important caveat: if there’s another subclassing on top of myWndProc(), restoring the window function to orgWndProc unhooks not only myWndProc(), but one or more other subclassings as well. While the wrongness of this will vary, it certainly isn’t right — a well-behaved subclassing should unhook only if it finds itself at the top of the food chain. The following condition must hold:
myWndProc == GetWindowLong( hwnd, GWL_WNDPROC );
Both the GWL_USERDATA technique and the porcupine technique share the limitation that each subclassing lives in ignorance of all others, so unhooking must proceed in reverse order of subclassing if either technique is to practice safe unhooking. This does not meet any reasonable definition of “unhooking at will.”
To allow separate parts of your code to subclass the same window and yet unhook in any order, subclassings must, unfortunately but necessarily, be aware of one another. To this end, wdjSUB maintains a linked list of subclassing descriptors for each subclassed window:
typedef struct SUBCLASSING {
WNDPROC wndProc;
WNDPROC wndProcSaved;
void *pData;
struct SUBCLASSING *pNext;
} SUBCLASSING;
I’ll get back to the details of this structure shortly. At any rate, subclassing is now a matter of attaching a new SUBCLASSING node to the head of the list and replacing the window function. Unhooking is a matter of removing a node from the list; restoration of the original window function takes place only when the head of the list is removed. The only thing I have to associate with the window is the head of the linked SUBCLASSING list.
This approach allows subclassing and unhooking with gay abandon, provided you stick to using the wdjSUB library. You’re still vulnerable to misbehaved “foreign” subclassings, though. When you’re ready to unsubclass a window, you should check first to see that the current window procedure is equal to your window procedure. If it isn’t, the implication is that someone else subclassed the window after you did. Hence, it’s quite possible that other subclasser thought your subclass procedure was the original window procedure and will, unbeknownst to you, later restore your subclass procedure when it unhooks. There’s really no way you can protect yourself against all the possible ways someone else’s subclass code could mess yours up, but you can be as defensive as possible.
wdjSUB does the best it can on the assumption that most subclassing schemes unhook on either WM_DESTROY or WM_NCDESTROY. wdjSUB provides automatic unhooking on both of these messages. If unhooking succeeds on WM_DESTROY, this opens the door for any subclassing below that wants to unhook on that message. If someone else “owns” the window at WM_DESTROY time, then wdjSUB waits to try again at WM_NCDESTROY time. If you just go ahead and restore the previous window procedure at WM_DESTROY time, then the code that subclassed the window after you will never get called again — it’s out of the loop. It’s likely that that code intended to perform some processing, such as freeing up memory or other resources, when it was unhooked. Taking someone else’s window subclass function out of the loop could result in resource leaks or worse problems.
If that someone else still hasn’t unhooked at WM_NCDESTROY time, then there’s little wdjSUB can do, since WM_NCDESTROY should be the last message that Windows sends before destroying the underlying window object.
Static Association
As mentioned previously, all that must be associated with a window is the head of its SUBCLASSING list. You may have noticed that this invalidates the killer criterion that shot down the GWL_USERDATA technique. I’m not going to resurrect it, though — it’s too risky for a general library. The remaining choices are to attach the list head to the window using a single window property or to maintain the mapping in a static table.
There is something very appealing and elegant about attaching the head of the SUBCLASSING chain to the window itself. There are drawbacks, though: window properties are exposed to the rest of the world, and it is within the realm of possibility that some Bad Person will steal your property. In other words, encapsulation is imperfect. Another drawback is that a poorly designed “foreign” subclasser could cause a window property leak (and other leaks). Suppose someone subclassed the window before you did and then decided to unhook long before receiving WM_DESTROY. If they rudely just restore the previous window procedure instead of noticing that someone else has subclassed the window, then your subclass procedure will be out of the loop and will never get called again. In this case, that means you won’t properly free up the window property you allocated.
You can avoid these problems by maintaining a static table of HWND/SUBCLASSING associations. This improves encapsulation and ensures that you can clean up after yourself. The data structure of choice here is a hash table. (Post-it notes have well-documented performance problems.)
What about multithreading? If you maintain a single global table, you must provide thread-safe access to the table itself. This is not a terribly good idea, because wdjCallOldProc() has an inner-loopy nature and thread synchronization is expensive. A better way to handle the problem is to maintain the table on a per-thread basis. Windows are thread-bound anyhow — you can subclass a window from any thread, but the subclassing function will execute in the context of the thread that created the window.
If creation and subclassing of a given window is the responsibility of a single thread, there is never any contention for the SUBCLASSING list, yet other threads are still free to send or post messages to the window. This, then, is a rule of wdjSUB programming, and to ensure compliance, both subclassing and unhooking are protected by an assertion:
assert( GetCurrentThreadId() ==
GetWindowThreadProcessId( hwnd, 0 ) );
The major disadvantage of the static table approach is more code and increased complexity. To keep the code size down, I decided to use a window property to store the SUBCLASSING list. This implementation detail does not affect the wdjSUB API.
The wdjSUB API
The interface to wdjSUB is defined by wdjsub.h (Listing
1). Some internal declarations are in internal.h (Listing
2), to make them available for debugging and testing, and the implementation
is in wdjsub.c (Listing 3).
The wndProc member of the SUBCLASSING structure serves to identify the subclassing. Although any unique value would serve as an identifier, using the address of the subclassing’s window function makes it easy for wdjUnhook() to check whether unhooking is safe.
In my original example, subclassing with wdjSUB would look like this:
wdjSubclass( myWndProc, GetDlgItem(
hwnd, ID_OF_EDIT_CONTROL_TO_BE_SUBCLASSED ), 0 );
This function returns TRUE on success, FALSE on failure. It fails if the window has already been subclassed using myWndProc(), if it fails to add a global atom, or if it can’t allocate enough memory for a SUBCLASSING structure.
The SubclassWindow() macro is defined in windowsx.h.
The application data parameter was set to zero in this example, so calling
wdjGetData( myWndProc, hwnd );
from myWndProc() would yield zero. The accompanying test program (described later) demonstrates real use of this facility.
Explicit unhooking is generally unnecessary. wdjSUB unhooks automatically on window destruction, provided wdjCallOldProc() is called for either WM_DESTROY or WM_NCDESTROY (preferably both):
wdjCallOldProc( myWndProc, hwnd, msg, wParam, lParam );
This function should be called only from myWndProc().
The allocPropertyName() and releasePropertyName() utility functions in wdjsub.c create and free the atom that names the property that holds the head of the window’s SUBCLASSING chain. Remember that atoms are reference-counted; multiple calls to GlobalAddAtom() require an equal number of calls to GlobalDeleteAtom() before the atom actually dies.
The find() utility function searches the SUBCLASSING list for a given node, while getHead() (a macro in internal.h) and setHead() handle all GetProp(), SetProp() and RemoveProp() calls.
The most important functions — wdjSubclass(), wdjUnhook(), and wdjCallOldProc() — are not as complex as they may appear; most of the code is devoted to tracing statements helpful during debugging or testing.
The Sample Program
wdjSUB comes with a sample program that lets you see what’s happening
under the hood (Figure 1). You’ll find the
test program in subtest.c (Listing 4) and
the dialog script in subtest.rc (Listing 5).
The test program lets you apply five different subclassings to a simple edit window. You can subclass and unhook these in any order, although unhooking will sometimes fail if the “Only unhook if expected wnd proc found” check box is checked. If it is not checked, subclassings may be lost instead.
Two of the subclassings (tracer() and upper()) are “foreign,” while the remaining three (digits(), hex(), and beep()) use wdjSUB. As long as you stick to the last three, you can do anything you like in any order. When you involve the first two, restrictions apply. If a foreign subclassing blocks a chain of wdj subclassings, for example, the head of the wdj chain is stuck for the duration of the blockage, but the others can safely be unhooked.
What happens if a foreign subclassing appears in between wdj subclassings? Try it and see; that’s what the test program is for.
The test program is in the privileged position of having perfect knowledge of all subclassing techniques employed. This lets updateStack() display the complete subclassing chain in the stack window.
The message trace window is used by the tracer() subclassing, and the diagnostics window provides a running commentary from wdjSUB and the test program.
The three wdj subclassings all demonstrate the association of subclassing-specific data with a window. Two of them store a string pointer, while the third stores a function pointer.
Finally, the “Destroy” button destroys the guinea pig; this lets you trace through automatic unhooking. Note that tracer() and upper() unhook on different messages!
All in all, this program should give you a feel for how subclassing chains work — and how they sometimes fail to work.
Conclusion
wdjSUB is far from being the final word about subclassing. I’ve sidestepped some issues and failed to consider some implementation alternatives (dynamically generated code stubs spring to mind). Still, wdjSUB is robust and simple, and covers (according to a complicated formula I just derived) 98 percent of your daily subclassing needs. If your needs fall into the other 2 percent, or if you have subclassing insights to offer, please drop me a line.
Petter Hesselberg is a Specialist Partner with Accenture's Oslo office. He's been programming Windows for the past thirteen years and is the author of Programming Industrial Strength Windows. He can be reached at petter.hesselberg@accenture.com.
