Robots.txt

User-agent: * Allow: / Disallow: /admin Disallow: /api Sitemap: https://www.nobrand.org.il/sitemap.xml # Crawl-delay Crawl-delay: 1 # Specific bots User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Slurp Allow: /

Search Engine Instructions

This robots.txt file tells search engines:

  • All pages are allowed to be crawled
  • Sitemap location for easy discovery
  • Specific permissions for major search bots (Google, Bing, Yahoo)

For Webmasters

Copy the robots.txt content above and save it as /robots.txt in your root domain. Then submit to Google Search Console and Bing Webmaster Tools.