User-agent: * Disallow: Disallow: /*.asp*$ Disallow: /*.cgi*$ Disallow: /*.htm*$ Disallow: /*.html*$