For Want of a Better Algorithm

It doesn't seem fair! How are schmucks like me supposed to go up against researchers from AT&T Labs and win anything? Here's what happened.

Netlfix, the online movie rental service, has a contest called the NetflixPrize with the goal of improving the accuracy of predictions about how much someone is going to like a movie, based on the viewer's movie preferences. And one prize was $50,000, which would keep me in lattes for the forseeable future.

So my algorithm went something like this: "If movie stars Sandra Bullock, then movie is bad. If movie stars Franka Potente, then movie is good -- really good."

Apparently that wasn't good enough for the folks at Netflix. I guess they wanted more details, which was what Team KorBell, a group of researchers at AT&T Labs Research, provided. Team KorBell consisted of Yehuda Koren, Robert Bell, and Chris Volinsky, who in their day job work on visualizing and analyzing large networks for AT&T. So right away you can probably guess they had an edge on me. Team KorBell improved upon the Netflix recommendation system by 8.43 percent -- the best score for the competition in which more than 27,000 contestants on more than 2,550 teams from 161 countries participated.

I won't go into the details of Team KorBell's solution. I'm too disgusted, and don't understand it anyway. However, you might want to take a look at their paper The BellKor Solution to the Netflix Prize. Let's just say that they included a lot more math than I did, and a whole bunch of equations and graphs. But supposedly it works.

Just to show that I'm not a sore loser, I'd like to say congratulations to Yehuda Koren, Robert Bell, and Chris Volinsky.

They may have walked away with the $50K, but the $1 million Grand Prize is still up for grabs and all I have to do is tweak my algorithm a bit -- say "Britney Spears = bad movie"?

-- Jonathan Erickson
jerickson@ddj.com

Posted by Jon Erickson at 11:36 AM  Permalink