User-agent: * allow: home.php$ allow: product.php$ Disallow: admin/ Disallow: provider/ Disallow: files/ Disallow: /cgi-bin/