| Por defecto todos los robots están: |
|
|
| Espera entre peticiones: |
|
| |
|
|
| Bots de Búsqueda: |
Google |
googlebot |
| |
MSN Search |
msnbot |
| |
Yahoo |
yahoo-slurp |
| |
Ask/Teoma |
teoma |
| |
GigaBlast |
gigabot |
| |
Scrub The Web |
scrubby |
| |
DMOZ Checker |
robozilla |
| |
Nutch |
nutch |
| |
Alexa/Wayback |
ia_archiver |
| |
|
| Bots Especiales: |
Google Image |
googlebot-image |
| |
Yahoo MM |
yahoo-mmcrawler |
| |
MSN PicSearch |
psbot |
| |
SingingFish |
asterias |
| |
Yahoo Blogs |
yahoo-blogs/v3.9 |
| |
|
| Directorios Restringidos: |
La ruta relativa a la raíz, debe contener una barra final "/" |
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|