# All robots will spider the domain User-agent: * Disallow: # Disallow directory /mail/ User-agent: * Disallow: /mail/ # Disallow directory /etc/ User-agent: * Disallow: /etc/ # Disallow directory /public_ftp/ User-agent: * Disallow: /public_ftp # Disallow directory /tmp/ User-agent: * Disallow: /tmp/