06-04-2008, 10:37 PM
|
 |
Lost Webmaster
|
|
Join Date: Jan 2007
Location: Petaling Jaya
Posts: 858
Rep Power: 40
|
|
robot.txt is a text file which tells the internet crawler to and not go to which part of ur hosting folder.
it helps u protect pages or files which u do not want to be listed in any search engine.
that's the brief info which i know about it...i m sure there are something else on this, but i dunno :P
|