Read My Lips

Every year, the list of things I'd like to accomplish gets longer, not shorter. Let's see: Rebuild an automobile motor (check, done that). Write a book (check, done that). Build a house (check, done that). Learn sign language (not yet, next year). Ride a motor scooter in Hyderabad (scratch that, not even on a bet). Learn lip-reading (not yet, next year). You get the idea.

As it turns out, researchers at the University of East Anglia in the UK are a step ahead of me and just might help me out. Led by Richard Harvey, a senior lecturer at UEA’s School of Computing Sciences, a research team is hopes to develop computer lip-reading systems that could be used for fighting crime, among other applications. The three-year project will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text. It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.

"We all lip read, for example in noisy situations like a bar or party, but even the performance of expert lip readers can be very poor," Harvey says. "It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill."

Harvey is teaming up with the Centre for Vision, Speech and Signal Processing at Surrey University, which has built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, which wants to investigate the feasibility of using the technology for crime fighting. According to Harvey, "The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence."

Another reason for developing computerized lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.

As for my interest, I'd bet that some interesting learning software will come out of this research -- software that I'll get to right after I climb a mountain somewhere.

Posted by Jon Erickson at 01:24 PM  Permalink