Don_G
(1000+ posts)
Send PM |
Profile |
Ignore
|
Thu Apr-01-04 11:59 AM
Original message |
Does "Robot.txt" Have Anything To Do |
|
With Air America?
I suppose #43 and Bill Gates has other ideals I'm not aware of beyuod DU and eBay.
|
smiley_glad_hands
(1000+ posts)
Send PM |
Profile |
Ignore
|
Thu Apr-01-04 12:03 PM
Response to Original message |
1. Robot.txt is a file placed on a server to tell |
|
Edited on Thu Apr-01-04 12:12 PM by vadem0557
the Google Web Crawler/Bot which pages not to index on a website--pages like xml forms and such that you don't want exposed publicly. On edit--exploiting sites that do not have a robot.txt file is termed: "google hacking". Also, where did you come across this file?
|
DU
AdBot (1000+ posts) |
Fri Apr 26th 2024, 11:32 PM
Response to Original message |