gmakstutis/openrag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

deploy: 900e5f9795

Browse source
This commit is contained in:
phact 2025-12-23 02:14:40 +00:00
parent 97645081e3
commit 93b197f711
1 changed files with 4 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
robots.txt
View file

@ -1,12 +1,8 @@
# Robots.txt for OpenRAG Documentation
# Block all crawlers by default
# Allow all crawlers
User-agent: *
Disallow: /
Allow: /
# Allow specific crawlers if needed (uncomment when ready for launch)
# User-agent: Googlebot
# Allow: /
# Sitemap location (uncomment when ready for launch)
# Sitemap: https://docs.openr.ag/sitemap.xml
# Sitemap location
Sitemap: https://docs.openr.ag/sitemap.xml