Email	Print Reprint
	add to:
	Del.icio.us Digg Google Furl	Slashdot Y! MyWeb Blink
	TABLE OF CONTENTS Threads: In a Nutshell Example 1: GUI Example 2: Sockets Example 3: Pipeline Stages

Example 1: GUI

GUI threads are a classic example of message pump-driven code. A GUI program is driven by queued events coming in from the user or elsewhere, and its code is organized as a set of responders providing the right thing to do in response to anything from the user pressing a button (e.g., Save, Print, search and replace) to a system event (e.g., window repaint, timer pulse, file system change notification). Many GUI systems offer priority queueing that lets later messages execute sooner even if other messages in the queue arrived earlier (e.g., SendMessage vs. PostMessage on Windows).

Consider the following simplified GUI pseudocode:

// Example 1: Sample synchronous GUI
// (not recommended)
//
while( message = queue.Receive() ) {
        // this could block
  if( it's a "save document" request ) {
    TurnSavingIconOn();
    SaveDocument();
       // bad: synchronous call
    TurnSavingIconOff();
  }
  else if( it's a "print document" request ) {
    TurnPrintingIconOn();
    PrintDocument();
      // bad: synchronous call
    TurnPrintingIconOff();
  }
else
  ...
}

Running nontrivial work synchronously on the GUI thread like this is a classic mistake, because GUIs are supposed to be responsive, and to be responsive requires not just responding to events but responding to them quickly. What if the GUI thread is busy processing a PrintDocument for several seconds, and in the meantime the user tries to resize or move the window? The resize or move request will be dutifully enqueued as a message that waits to be processed in its turn once the printing is complete; but in the meantime the user sees no effect and may try the same action, or give up and try something else instead, only to have all of the potentially duplicated or contradictory commands performed in sequence when the GUI thread is finally able to service them -- often with unintended effects. Synchronous execution leads to poor responsiveness and a poor user experience.

Threads that must be responsive should never execute high-latency work directly. This includes not just work that may take a lot of processing time, but also work that might have to wait for another thread, process, or computer -- including communications or trying to acquire a lock.

There are three major ways we can run high-latency work synchronously to move it off a thread that needs to stay responsive. Using the GUI thread as a case in point, let's examine them in turn and consider when each one is appropriate.

Async Option 1: Dedicated Background Thread

Our first option for moving work off the GUI thread is to execute it instead on a dedicated background worker thread. Because there is exactly one background worker for all the GUI-related grunt work, one potentially desirable effect is that the background work will be processed sequentially, one item at a time. This can be a good thing when two pieces of work would conflict (e.g., want to use the same mutable data) and so benefit from running sequentially; it can be a drawback when earlier tasks block later tasks that could have run sooner independently (see Option 2 below). Note also that messages don't have to go only one way: The asynchronous work can send back notifications ranging from a simple "I'm done" to intermediate information like progress status.

Figure 1 shows an example of this arrangement, where the user presses Print (dark blue) and then Save (light blue), and the two pieces of work are delegated to the worker where they can run asynchronously while the user continues to perform other operations such as moving windows or editing text (gray).

Let's consider two common ways to express the code for this option. First, we can arrange for the GUI thread to send tags representing the work that needs to be done:

// Option 1(a): Queueing work tags for a dedicated
// background thread. Suitable where tasks need
// to run sequentially with respect to each other (note
// corollary: an earlier task can block later tasks).

// GUI thread
//
while( message = queue.Receive() ) { // this could block
   if( it's a "save document" request ) {
      TurnSavingIconOn();
      worker.Send( new SaveMsg() ); // send async request
   }
   else if( it's a "save document" completion notification ) {
     TurnSavingIconOff();// receive async
// notification
   }
   else if( it's a "print document" request ) {
      TurnPrintingStatusOn();
      worker.Send( new PrintMsg() ); // send async request
   }
   else if( it's a "print document" progress notification ) {
      if( percent < 100 ) // receive async
// notification
         DisplayPrintPercentComplete( percent );
      else
         TurnPrintingStatusOff();
   }
   else
   ...
}
// Dedicated worker thread: Just interprets
// the tags on its queue and performs the
// appropriate action for each tag.
//
while( message = workqueue.Receive() ) {// this could block
   if( it's a "save document" request )
      SaveDocument(); // now sends completion
                                        // notification
   else if( it's a "print document" request )
      PrintDocument(); // now sends progress
                                        // notifications
   else
      ...                               // etc., and check for
                                        // termination
}

Alternatively, instead of sending tags we can send executable messages, such as Java runnable objects, C++ function objects or lambdas, C# delegates or lambda expressions, or even C function pointers. This helps simplify the worker thread mainline:

// Option 1(b): Queueing runnable work for a dedicated
// background thread. Suitable where tasks need
// to run serially with respect to each other (note
// corollary: an earlier task can block later tasks).

// GUI thread
//
while( message = queue.Receive() ) { // this could block
   if( it's a "save document" request ) {
      TurnSavingIconOn();
      worker.Send( [] { SaveDocument(); } ); // send async work
   }
   else if( it's a "print document" request ) {
      TurnPrintingStatusOn();
      worker.Send( [] { PrintDocument(); } );  // send async work
   }
   else if( it's a "save document" notification ) { ... }
                                       // as before
   else if( it's a "print document" progress notification ) { ... }
                                       // as before
   else
      ...
}
// Simplified dedicated worker thread:
// Just executes the work it's being given.
//
while( message = workqueue.Receive() ) { // this could block
   message();                // execute the given
                                         // work
   ...                                   // check for
                                         // termination
}

Async Option 2: Background Thread Per Task

Our second option is a variant of the first: Instead of having only one dedicated background worker, we can choose to launch each piece of asynchronous work as its own new thread. This makes sense when the work items really are independent and won't interfere with each other, though it can mean that work that is launched later can finish earlier, as illustrated in Figure 2.

Here's sample code that sketches how we can write this option, where red code again highlights the code that is different from the previous example.

// Option 2: Launching a new background thread
// for each task. Suitable where tasks don't need
// to run sequentially with respect to each other.
//
while( message = queue.Receive() ) { // this could block
   if( it's a "save document" request ) {
      TurnSavingIconOn();
      ...  new Thread( [] { SaveDocument(); } ); // run async request
   }
   else if( it's a "print document" request ) {
      TurnPrintingStatusOn();
       new Thread( [] { PrintDocument(); } );// run async request
   }
   else if( it's a "save document" notification ) { ... }
                                      // as before
   else if( it's a "print document" progress notification ) { ... }
                                      // as before
   else
      ...
}

The ellipsis before new Thread stands for any housekeeping we might do to keep track of the threads. In languages with garbage collection we would normally just launch a new Thread and let it go; otherwise, without garbage collection we might maintain a list of launched threads and clean them up periodically.

Async Option 3: Run One-Shot Independent Tasks On a Thread Pool

Thread pools are about expressing independent work that will get run on a set of threads whose number is automatically chosen to match the number of cores available on the machine. Pools are mainly intended to enable scalability (Pillar 2), but can sometimes also be used for running what would otherwise be short threads.

Figure 3 shows how a short-running thread that rarely waits/blocks can instead "rent-a-thread" to run as a pool work item. Again, this technique is for simple one-shot work only, and it's very important not to run work on a thread pool if the work could block, such as wait to acquire a mutex or wait to receive a message (sending out messages is okay because that doesn't block).

Here's some sample code, which again should be considered pseudocode given that the specific spelling of "run this work on that pool" varies from one language and operating system to another:

// Option 3: Launching each task in a thread pool.
// Suitable for one-shot independent tasks that
// don't block for locks or communication.
//
while( message = queue.Receive() ) {   // this could block
   if( it's a "save document" request ) {
      TurnSavingIconOn();
      pool.run( [] { SaveDocument(); } ); // async call
   }
   else if( it's a "print document" request ) {
      TurnPrintingStatusOn();
      pool.run( [] { PrintDocument(); } ); // async call
   }
   else if( it's a "save document" notification ) { ... }
                                        // as before
   else if( it's a "print document" progress notification ) {  ... }
                                       // as before
   else
      ...
}

A Word About OpenMP and Work-Stealing Runtimes

Table 1 shows two other tools that are available (or becoming available) to express potentially asynchronous work, but for a different purpose:

• Work-stealing runtimes. Work stealing, pioneered by Cilk [3], is the engine behind the emerging crop of next-generation runtimes that target providing scalable concurrency (Pillar 2). They make it easy and efficient to express work that might be done in parallel if there are enough hardware resources to run it in parallel, yet add hardly any overhead if there aren't extra cores handy because the default implementation is to run the work sequentially in the original thread. Products based on work stealing include Intel's Threading Building Blocks [4], and at least four upcoming products: the Visual C++ 2010 Parallel Patterns Library (PPL) [5], the .NET 4.0 Task Parallel Library (TPL) [6] and Parallel LINQ (PLINQ) [7], and the Java 7 Fork/Join framework [8].
• OpenMP. Long available and now in version 3.0, OpenMP is all about parallelizing Fortran- and C-style loops, running them on a sort of thread pool under the covers to get data-parallel scalability. [9]

I mention these tools here primarily to say that they're meant for Pillar 2, as Table 1 shows. They're all about different ways to split and subdivide work across available cores to get the answer faster on machines having more cores, from doing loop iterations in parallel, to working on subranges of the data in parallel, to performing recursive decomposition (divide-and-conquer algorithms) in parallel.

These are not the right tools for Pillar 1, which means that they neither compete with threads nor replace them. The reason that work-stealing runtimes and OpenMP are unsuitable for running work asynchronously is because the work is not guaranteed to actually run off the original thread -- i.e., it's not guaranteed to be asynchronous at all.

For completeness, though, here's a taste of the syntax:

// Option 4 (NOT recommended for PrintDocument):
// Run on a work stealing or OpenMP runtime.
// Using Visual C++ 2010 Parallel Patterns Library (PPL)
// with the convenience of ISO C++0x lambdas
//
taskgroup.run( [] { PrintDocument(); } );
// Using .NET 4.0 Task Parallel Library (TPL)
// with the convenience of C# lambdas
//
Parallel.Invoke( () => { PrintDocument(); } );
// Using Java 7 ForkJoin with an explicit runnable object
//
class PrintTask extends ForkJoinTask {
   public void run() { PrintDocument(); }
   ...
}
...
fjpool.execute( new PrintTask(); );

OpenMP is not based on work stealing, but does offer a similar construct with the same caveat:

// Using OpenMP (again, NOT recommended for PrintDocument)
//
#pragma omp task
{
   PrintDocument();
}

Previous Page | 1 Threads: In a Nutshell | 2 Example 1: GUI | 3 Example 2: Sockets | 4 Example 3: Pipeline Stages Next Page

RELATED ARTICLES Petascale Tools and Genomic Evolution Legal Compliance: Open Source and Quality Assurance SEI, Army Team Up on Software Testing Highly Scalable Parallel Sorting SC09 Storage Challenge Award		TOP 5 ARTICLES No Top Articles.