Commit graph

3403 commits

Author SHA1 Message Date
Geoff-Robin
a3fbbdf8eb Solved nitpick comments 2025-10-08 14:58:02 +05:30
Geoff-Robin
af71cba07f Trying to resolve uv.lock 2025-10-08 14:11:43 +05:30
Geoff-Robin
8d27da659a removed dotenv imports 2025-10-08 14:07:01 +05:30
Geoff-Robin
ea33854d49 Removed print statement logging and used cognee inbuilt logger and updated doctrings. 2025-10-08 14:06:13 +05:30
Geoff-Robin
49858c5416 Made api_key field in TavilyConfig models to be Optional[str] type to allow None value 2025-10-07 22:38:13 +05:30
Geoff-Robin
f59c278ae9 Added await 2025-10-07 22:36:24 +05:30
Geoff-Robin
0fd55a737f ruff formatted 2025-10-07 22:16:02 +05:30
Geoff-Robin
fc660e4027 Closed crawler instance in a finally block 2025-10-07 22:15:06 +05:30
Geoff-Robin
fcd91a9709 Added self as an argument to all previous methods that were static methods 2025-10-07 21:51:26 +05:30
Geoff-Robin
3d53e8d6f1 Removed print statement that I used for debugging 2025-10-07 20:59:19 +05:30
Geoff-Robin
d91ffa2ad6 Removed staticmethod decorator from bs4_crawler.py, kwargs from the function signature in save_data_item_to_storage.py, removed unused imports in ingest_data.py and added robots_cache_ttl as a config field in BeautifulSoupCrawler. 2025-10-07 20:56:23 +05:30
Geoff-Robin
fdf85628c7 Added uv.lock again 2025-10-07 01:40:19 +05:30
Geoff-Robin
f71cf774d2 . 2025-10-07 01:34:40 +05:30
Geoff-Robin
902f9a3b6a Changed cognee-mcp\pyproject.toml 2025-10-07 01:26:09 +05:30
Geoff-Robin
b5a1957b0f Regenerate uv.lock after merge 2025-10-07 01:22:39 +05:30
Geoff-Robin
5dcd7e512f Changes uv.lock 2025-10-07 01:09:41 +05:30
Geoff-Robin
1f36dd3d71 Solved nitpick comments 2025-10-06 19:44:54 +05:30
Geoff-Robin
54f2580f2d Solved more nitpick comments 2025-10-06 19:02:11 +05:30
Geoff-Robin
1c0e0f0fe1 Solved more nitpick comments 2025-10-06 18:32:10 +05:30
Geoff-Robin
d4ce340cb5 Removed unused imports 2025-10-06 18:31:08 +05:30
Geoff-Robin
7fe1de770d Remove assignment to unused variable graph_db' 2025-10-06 18:29:58 +05:30
Geoff-Robin
0a9b624010 changed return type for fetch_page_content to Dict[str,str] 2025-10-06 18:27:54 +05:30
Geoff-Robin
3c9e5f830b Solved more nitpick comments 2025-10-06 18:16:31 +05:30
Geoff-Robin
791e38b2c0 Solved more nitpick comments 2025-10-06 18:00:20 +05:30
Geoff-Robin
1b5c099f8b CodeRabbit reviews solved 2025-10-06 17:15:25 +05:30
Geoff-Robin
ae740eda96 Added related documentation 2025-10-06 04:23:10 +05:30
Geoff-Robin
667bbd775e Added cron job and removed obvious comments 2025-10-06 04:12:32 +05:30
Geoff-Robin
4d5146c802 Added Documentation 2025-10-06 04:00:15 +05:30
Geoff-Robin
0f64f6804d Done adding cron job web scraping 2025-10-06 03:45:09 +05:30
Geoff-Robin
e5633bc368 corrected F402 error pointed out by ruff check 2025-10-06 03:44:24 +05:30
Geoff-Robin
f449fce0f1 Done with scraping_task successfully 2025-10-06 02:27:20 +05:30
Geoff-Robin
f148b1df89 Added support for multiple base_url extraction 2025-10-05 20:13:44 +05:30
Geoff-Robin
77ea7c4b1d Added APScheduler 2025-10-05 20:02:02 +05:30
Geoff-Robin
c2aa95521c removed structured argument 2025-10-05 20:00:19 +05:30
Geoff-Robin
2cba31a086 Tested and Debugged scraping usage in cognee.add() pipeline 2025-10-04 21:26:25 +05:30
Geoff-Robin
ab6fc65406 Added global context for bs4crawler and tavily config 2025-10-04 19:40:37 +05:30
Geoff-Robin
da7ebc4574 Removed asyncio import 2025-10-04 15:10:46 +05:30
Geoff-Robin
fbef6675bc removed unused Dict import from typing 2025-10-04 15:10:05 +05:30
Geoff-Robin
20fb77316c Done with integration with add workflow when incremental_loading is set to False 2025-10-04 15:01:13 +05:30
Geoff-Robin
1ab9d24cf0 Changed bs4_connector.py to bs4_crawler.py 2025-10-03 12:33:13 +05:30
Geoff-Robin
edd119ef97 first iteration of bs4_connector.py done 2025-10-02 22:04:50 +05:30
Geoff-Robin
4979f43fc0 Added playwright as a dependency 2025-10-02 02:21:33 +05:30
Geoff-Robin
c283977035 switched httpx AsyncClient to fetch webpage 2025-10-02 02:01:46 +05:30
Geoff-Robin
60499c439c Added logging 2025-10-02 01:54:56 +05:30
Geoff-Robin
925bd38195 Setup models.py and utils.py 2025-10-02 01:32:00 +05:30
Geoff-Robin
70a2cc9d65 removed scrapy and added bs4 2025-10-02 01:28:48 +05:30
Geoff-Robin
6348c9d8de Created models.py 2025-09-30 20:46:26 +05:30
Geoff-Robin
510926f56c included scraping dependencies 2025-09-30 20:39:04 +05:30
Vasilije
18dcab3cac
Update README with cognee features and deployment info
Clarify the functionality and deployment of cognee.
2025-09-29 13:37:09 +02:00
Vasilije
107b5af6b5
Feature/cog 2979 fix falkordb adapter (#1430)
<!-- .github/pull_request_template.md -->

## Description
<!-- 
Please provide a clear, human-generated description of the changes in
this PR.
DO NOT use AI-generated descriptions. We want to understand your thought
process and reasoning.
-->
Falkordb adapter didn't work on main repo, but we have it working on
community. Decision was to remove it from main repo, so it is removed.

## Type of Change
<!-- Please check the relevant option -->
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Code refactoring
- [ ] Performance improvement
- [ ] Other (please specify):

## Changes Made
<!-- List the specific changes made in this PR -->
- 
- 
- 

## Testing
<!-- Describe how you tested your changes -->

## Screenshots/Videos (if applicable)
<!-- Add screenshots or videos to help explain your changes -->

## Pre-submission Checklist
<!-- Please check all boxes that apply before submitting your PR -->
- [ ] **I have tested my changes thoroughly before submitting this PR**
- [ ] **This PR contains minimal changes necessary to address the
issue/feature**
- [ ] My code follows the project's coding standards and style
guidelines
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have added necessary documentation (if applicable)
- [ ] All new and existing tests pass
- [ ] I have searched existing PRs to ensure this change hasn't been
submitted already
- [ ] I have linked any relevant issues in the description
- [ ] My commits have clear and descriptive messages

## Related Issues
<!-- Link any related issues using "Fixes #issue_number" or "Relates to
#issue_number" -->

## Additional Notes
<!-- Add any additional notes, concerns, or context for reviewers -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-09-28 17:02:48 +02:00