The Little Engines That Couldn't
When users type a query into a search engine and click on the "search" button, they assume the engine will check the entire Web to satisfy their request. But according to a recent study by the NEC Research Institute in Princeton, New Jersey, which analyzed the searching abilities of the most popular engines, it seems they don't even come close. In fact, even the most comprehensive search engine available, HotBot, only covers about 34 percent of the Web's estimated 320 million pages of information.
Search engine creators are not to blame for this shortcoming, say researchers. It's the Web itself, which is growing so rapidly, no one can track the changes. To expand a search, use a meta-search engine such as go2net Inc.'s MetaCrawler (http://www.metacrawler.com), which sends queries to six different search engines and collates the results.