# Robots.txt for OpenRAG Documentation # Block all crawlers by default User-agent: * Disallow: / # Allow specific crawlers User-agent: Googlebot Allow: / # Sitemap location Sitemap: https://docs.openr.ag/sitemap.xml