Search engines use spider (sometimes called crawler) software to locate, retrieve, and index data from a Web page. Typically, spiders do not travel deep within a site. Moreover, they do not index all Web pages, or even all data on page.
The Best:
AltaVista
AlltheWeb
Google
iLOR
Lycos
AOL.COM Search
Argos
Ask Jeeves
Excite
Gigablast
Overature
Hotbot
Lookle
MSN Search
Northern Light
Search123
Teoma Search
WISEnut