Spamdexing:

Also called search engine spamming.  It combines techniques employed by some Web marketers and designers to fool a search engine’s spider and indexing programs to ensure that their website always appears at or near the top of the list of search engine results.

I’m sure you’ve come across this without realizing it.  You go to Google [...]

Crawler (also Spider, Robot or Bot):

Component of a search engine that gathers listings by automatically trolling the Web and following links to webpages.  It makes copies of the webpages found and stores them in the search engine’s index.

The faster the crawlers work, the faster the pages are indexed and show up in search engine results [...]

Spider:

A software program that crawls the Internet by following links and indexing webpages.

Spiders are the basis of all search engines.  Google, Yahoo, and MSN utilize these programs to index the content throughout the net.  This is why the debate between nofollow and dofollow links and their effect on search indexing rages on.
____
Glossary Definition From ABC’s [...]