Quantcast
Channel: Linksite Blog » website
Viewing all articles
Browse latest Browse all 4

What is a Spider, Bot or Robot?

0
0

A “Spider, Bot, Robot, or Crawler” is an automated program or script operated by a search engine that surfs the Web just like you would. As it visits each Web site, it records or indexes it, all the words on each site and notes each link to other sites.

It then “clicks” on a link, and continues to read, index and store another Web site.
The Spider often reads and then indexes the entire text of each Web site it visits into the main database of the search engine it is working for. (Some engines indexing up to a certain number of pages of a site, often about 400, and then stopping. Apparently, this is because the Web has become so large that it’s unfeasible to index everything. How many pages the spider will index is not entirely predictable. Therefore, it’s a good idea to specifically submit each important page in your site that you want to be indexed, such as those that contain important keywords).

A Spider is like an electronic librarian who cuts out the table of contents of each book in every library in the world, sorts them into a gigantic master index, and then builds an electronic bibliography that stores information on which texts reference which other texts. Some Spiders can index over a million documents a day! For your marketing purposes, you should know that some search engines have spiders that will visit your Web site. What the spider sees on your site will determine how your site is listed in its index.
Remember, as a software spider visits your site it notes any links on your page to other sites. In any search engine’s vast database are recorded all the links between sites. The search engine knows which sites you linked to, and more importantly, which ones linked to you.
Many engines will even use the number of links to your site as an indication of popularity, and will then boost your ranking based on that factor.


Viewing all articles
Browse latest Browse all 4

Latest Images

Trending Articles





Latest Images