Last week we gave the webinar scene a break. Instead, Tracey talked me into going with her to the new Bill Gates Hillman Complex located on Carnegie Mellon campus in Pittsburgh. Holger Hoos from the University of British Columbia was giving a talk called "Taming the Complexity Monster". I figured how could we go wrong!

As I sit at my computer with it multicores considering the advantages of parallelism, faster computers, better performance, a strange feeling comes over me, 'Haven't I heard this before?'

Of course we're a little jealous of those developers who get to develop those fun and nifty IPhone apps! Perhaps we're just a tad bit curious too. But for the moment we are absolutely in the grips of a very different kind of software development.

On one hand it's funny. On the other hand, well ... it's funny. It's probably a matter of poetic justice being served up. Something I did or something Tracey did in a past life. But recently we seem unable to escape conversations that end up in questions (which we normally evade) about what we do.

We struggle to think in parallel and develop models and paradigms that imitate behavior with innate parallelism. Peer-to-peer, workpile, boss worker, consumer producer describe models for concurrent work and communication. But are these models sufficient to model the behavior of complex systems that utilize massive parallelism?

First a disclaimer or at the very least an acknowledgment. We are heavy consumers of Mac technology at Ctest Labs. Our Pantheon cluster boasts several fully loaded multicore Macs. We combine Macs with Sun boxes and Linux boxes in a way that makes for a very formidable cluster-based supercomputer that we call the Pantheon. So right off the bat let me say that we have much love for the Mac.

Is it called the server room? Computer operations? The rack closet? Joe's office? I've lost track. Seems like there's a different name for it wherever I go -- the place in the organization where all the heavy duty (important) computers are kept.

There are several concurrency schemes that describe how parallelism can be performed. Concurrency schemes such as peer-to-peer, boss-worker, workpile, and pipeline describes how tasks distribute work in parallel. SIMD (Single Instruction Multiple Data) and MIMD (Multiple Instructions Multiple Data) are concurrency schemes that achieve data level parallelism.

Once a software paradigm has reached a certain level of saturation and it is understood enough, some vendor is going to produce a template or a commercial framework that captures some of the key basics of that paradigm in a box or in a easy-to-use set of tools. At that point that entire software paradigm is considered solved.

Do you remember flowcharts? Flowcharts were/are a crucial tool in developing the flow of the logic in a structured program, process, an algorithm, etc. I remember how excited I was when I bought my first green plastic flowchart template. Some of my flowcharts were pages long but the process of flowcharting always clarified my logic .

This I believe: "Parallel Programming is Fundamental"

So - next week, I'm part of a small group heading off to put actions to these thoughts and teach parallel programming for three days to high school students and some high school teachers.

We all know that trying to design and understand the behavior of programs that contain concurrency/parallelism is more difficult than understanding single-threaded programs.

Moore's Law, Amdahl's Law, Murphy's Law. It's getting to be a bit much.

The popular paradigm of the computer is a web client, and large chunks of the software development effort is connected to web development. But Tracey and I are still trapped with the notion of the computer as a problem solver, not as part of the solution but as a solver. We use the computer to solve hard problems and we currently are trying to solve unsolvable problems.

So maybe there is hope for the Sun worshipers after all. In a recent interview (Reuters May 7, 2009), Oracle's Larry Ellison made a commitment to further SPARC technology. "We want to work with Fujitsu to design advanced features into the SPARC microprocessor aimed a improving Oracle database performance".

You can get five programmers in a room to give you six different definitions of parallelism. I've been in situations where a bunch of programmers were throwing the terms "parallel programming" and "multithreading" around assuming that they were all on the same page. I have to be careful because these days even the term "programmer" carries a certain amount of controversy, depending on the company one keeps.

I guess it all depends on where you jump into the fray with this parallelism stuff. In the server world, most of the applications are concerned with transaction throughput and the number of simultaneous user requests that can be successfully processed. So various schemes are concocted to deal with the load of simultaneous user requests. So multicore configurations have an obvious application at the server level. In the application world, things are not always so clear, but requirements for concurrency pop up.

Surely some vendor should be able to come along and make the number of cores or processors transparent to our application. After all, isn't one of the operating system's jobs to hide the details of the hardware from the programmer?

GPUs are dedicated processors that perform 3D graphics rendering. GPU manufactures are in a frenzy to deliver the latest technology that accelerates performance at a lower cost while still delivering high-end 3D graphics. Maybe this is something to think about. Maybe not. I do.

At Ctest Labs, we have (or had?) really big plans for the Sun Ultrasparc T2 and maybe Niagra 3. These are very nice boxes and fit our multicore needs just fine. But now that Oracle is acquiring Sun, what will become of the Ultrasparc T1, T2, and that whole line of multicore computers that had CMT?

Of course, the assumption is the answer is somewhere in the Cloud, right? We all know the Cloud has all the answers. But what if the Cloud doesn't have the answer?

Whether your problems are computationally intense or computationally large, maybe multicores can help manage the possible solutions. In guessing our 12-character code (three letter prefix and the 9 remaining characters of any arrangement of letters and digits, with replacement), we are dealing with a very large search space; an exhaustive search requiring 26³ x 36⁹ possibilities to consider. How could we interject parallelism into solving such a problem?

It really all depends on what we are referring to when we ask the software agent to solve the riddle of which tastes better coke or pepsi?. In the typical user-web-browser-search-engine-interaction, the onus for all of the parallel processing is on the user.

After all, what does Project Purity have to do with Megaton and Vault 101? Now that you mention it, who watches the watchmen? How will we know when the Singularity is near? All important questions in some domain. Each question hiding an even more fundamental problem.