User-agent: * #allows all search engine spiders. Disallow: /bhAdmin/ #disallow them to crawl admin folder.