Spider

In the context of web design, a spider is not an eight-legged creepy crawly. It’s a program that surfs the Internet, jumping from one page to the next by following links just like a human would. The spider’s job is to store (“index”) all the information it finds. A search engine like Google makes use of spiders to collect information from and about web sites to facilitate its web search.