cognee/cognee/tasks/web_scraper
2025-10-21 23:11:01 +01:00
..
__init__.py lazy load cron_web_scraper_task and web_scraper_task 2025-10-21 23:11:01 +01:00
config.py validate e2e - urls are saved as htmls, and loaders are selected correctly 2025-10-21 22:47:52 +01:00
default_url_crawler.py validate e2e - urls are saved as htmls, and loaders are selected correctly 2025-10-21 22:47:52 +01:00
models.py Addressed code rabbit comment on shortening content 2025-10-08 16:51:03 +05:30
types.py validate e2e - urls are saved as htmls, and loaders are selected correctly 2025-10-21 22:47:52 +01:00
utils.py validate e2e - urls are saved as htmls, and loaders are selected correctly 2025-10-21 22:47:52 +01:00
web_scraper_task.py Separate BeautifulSoup crawling from fetching 2025-10-21 22:47:52 +01:00