Email	Print Reprint
	add to:
	Del.icio.us Digg Google Furl	Slashdot Y! MyWeb Blink

When Henry Ford created the Model T, he never could have imagined where Ferruccio Lamborghini would one day take his concept. That's the way it works in business: Good ideas tend to precede good technology. Search engines are a classic example. What many old-fangled engines often give is a feast that feels like a famine—a giant plate of options that often fails to deliver what you need. "You're either getting way too much, and you have to figure out what to do with it and where to go next," says Tony Frazier, vice president for product management of search company iPhrase (www.iphrase.com), "or you're not getting anything at all."

A report issued by Forrester Research in September 2002 concluded soberly, "Most companies already own a search engine—one that doesn't work." Indeed, says Paul Sonderegger, the analyst who wrote the study, "Part of the problem may still be that users still can't find what they're looking for, and when that happens they abandon the search." That notion is backed by a WebTop study that found that American computer users experience "search rage" if they don't find what they want within twelve minutes. The problem, of course, is that many search engines are ill-suited to discern what you're really looking for among the ever-expanding, Himalayan-sized mountains of data.

The good news is that the search-engine arena is wildly competitive, with nearly a hundred solutions on the market from companies ranging from upstart Endeca (www.endeca.com) to behemoths like Google, Inktomi, and Microsoft. So you can bet that improvements are already on the horizon. "There are some real hot spots of innovation that get well beyond the old Boolean method," Sonderegger says, "and there is enormous demand right now for information retrieval technology."

A good search engine can help get more out of the investments you've already made, and can help you hang on to users. The next generation of search engine technology can discern meaning or intent. It can understand the language you speak, see what you see, and gather information from you about what you need. The caveat is that in such a competitive field, not all of the vendors will survive.

Beyond Boolean

The old search engine ways are just not working. There are obvious flaws with ranking pages by relevance—keyword spamming being the most prominent. And listing pages by popularity, Google's current system, all but rules out pages that might be valuable but collect few return links. Perhaps the most heated race at the moment is to create a search engine that goes beyond the timeworn concept of ranking pages by popularity or relevance alone. That means developing a search engine that not only understands what you want, but also can reach into multiple platforms—your email or your intranet, for example—and deliver results in a clear way. It also means the ability to decipher ambiguous searches, recognize natural language, and deploy sophisticated multifactor search algorithms.

Questions about accessing information from different locations and devices have proved to be among the most vexing, forcing employees to waste time bouncing between company intranets and browsers, with no interface. Among companies making inroads in that realm is Divine (www.divine.com), which in June 2002 began offering SinglePoint Search, a tool with an open architecture that enables users to search all of their resources, in any format, simultaneously. "If you're a global manufacturing company with fifty offices and thousands of employees using dozens of intranet sites, it can be pretty daunting to try to connect everything together," says Sid Probstein, SinglePoint's product manager. "You could be trying to locate pages on different intranets in different languages. The idea is to get the content you want all in the right place, and take advantage of all the avenues you have available to get what you want."

SinglePoint includes classification technology that helps users find relevant data by organizing information around hierarchical topics—presenting them in logical groups. Every document is classified automatically according to a seventeen thousand-node customizable taxonomy that Probstein says saves thousands of hours of development time. And a multifactor relevance-rank algorithm is applied to all content to help identify the most desirable results, regardless of source.

How does it work? Say a user is looking for information on Shakespeare's character Falstaff. SinglePoint looks at relevance—how many times each keyword is mentioned—but also ranks by length of document (a long one is likely more valuable than a very short one), by timeliness (one week old is more valuable than two years old), by inverse database frequency (a document that mentions Falstaff more than Shakespeare is more valuable than the inverse), and by prominence and positioning of the keywords (if both Shakespeare and Falstaff are in the title, the document is probably highly relevant).

SinglePoint also weeds out ambiguity from queries. Type in "bond," and the engine will pose the question, "What do you mean by bond?" and list a series of options, like "financial," "chemical," and "James." Each subject area has a folder the user can open that helps further define it: open "financial" and the options include links for mutual funds, index funds, and other related material.

One Step, a product created by iPhrase, offers similar capabilities: It uses an algorithm based on natural-language phrases. "Unless you understand what the users are asking, you don't have much of a chance of making the search any more relevant," says André Pino, iPhrase's senior vice president for marketing. The company's clients include Charles Schwab, LexisNexis, and Gateway. "There's no silver bullet here. What we're trying to do is get a better, more humanized interaction between the user and the search engine."

1 | 2 | 3 Next Page

RELATED ARTICLES Q&A: Software Testing In A Virtualized World Secure Software Needs Careful Testing -- And Lots Of It Nozzle: Counteracting Memory Exploits SDK, .NET Controls Released for Communications Enabling Web Pages Welcome To the World of Cryptographic Voting		TOP 5 ARTICLES No Top Articles.