Commit graph

470 commits

Author SHA1 Message Date
Boris
6e5acec292
refactor: make run_pipeline a high-level api for running pipelines (#1294)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-27 09:49:20 +02:00
Boris
007399b1c6
feat: migrate pipeline status reset to add pipeline (#1289)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-26 19:06:08 +02:00
Boris
ea4f58e8fa
feat: migrate pipeline input validation to a layer (#1284)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-26 18:25:48 +02:00
Igor Ilic
5771b36c4c
feat: cognee pipeline layers (#1287)
<!-- .github/pull_request_template.md -->

## Description
Add dataset authorization layer
Add pipeline processing status layer

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-26 16:35:05 +02:00
Igor Ilic
0d77649716 Merge branch 'cognee-pipeline-layers' of github.com:topoteretes/cognee into cognee-pipeline-layers 2025-08-26 15:59:32 +02:00
Igor Ilic
5ea3056564 refactor: Rename auth layer 2025-08-26 15:58:53 +02:00
hajdul88
d91b0f6aa3
feature: adds pipeline execution layer to cognify (#1291)
<!-- .github/pull_request_template.md -->

## Description
feature: adds pipeline execution layer to cognify

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-26 14:07:19 +02:00
Igor Ilic
03fc3c82f9
Merge branch 'dev' into cognee-pipeline-layers 2025-08-26 14:05:18 +02:00
Igor Ilic
950d29a678 refactor: Update typing 2025-08-26 14:04:33 +02:00
Igor Ilic
9a4e8ddc4d refactor: Make the authorized_user_datasets function more understandable 2025-08-25 17:22:37 +02:00
Igor Ilic
d4b23aa565 feat: Add process_pipeline_check 2025-08-25 17:16:08 +02:00
hajdul88
6f230c5a38
feature: adds environment_setup_and_checks general purpose layer to cognee_pipeline (#1283)
<!-- .github/pull_request_template.md -->

## Description
feature: adds environment_setup_and_checks general purpose layer to
cognee_pipeline

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-25 16:20:49 +02:00
Igor Ilic
6d438c8800 feat: Add dataset and pipeline status layer 2025-08-25 12:47:48 +02:00
hajdul88
f5d8fc6e81 chore: ruff ruff 2025-08-19 16:50:50 +02:00
hajdul88
4a5d5f70d0 feat: adds feedback weights to edges 2025-08-19 16:50:21 +02:00
hajdul88
c6ec22a5a0 feat: adds scores to Feedback node 2025-08-19 13:36:22 +02:00
hajdul88
0fbe218eef chore: fixes ruff 2025-08-18 18:36:04 +02:00
hajdul88
fc43ac7a01 feat: adds user feedback search type 2025-08-18 17:54:49 +02:00
hajdul88
b6be61776a fix: fixes tests 2025-08-18 13:50:21 +02:00
hajdul88
d53ebb2164
Merge branch 'dev' into feature/cog-2734-cognee-feedbacks-interactions-poc-to-prod 2025-08-18 13:17:13 +02:00
hajdul88
711c805c83 feat: adds cognee-user interactions to search 2025-08-18 13:14:06 +02:00
Vasilije
c4ec6799a6
Merge branch 'dev' into move_to_gpt5 2025-08-17 12:20:57 +02:00
vasilije
b0e3f89340 move to gpt5 2025-08-17 12:19:34 +02:00
Vasilije
73e1aa768c
Merge branch 'dev' into issues/1183 2025-08-15 15:05:00 +02:00
Daulet Amirkhanov
8d045b942e chore: forgot to remove NetworkX check in NLPRetriever 2025-08-15 10:56:19 +01:00
Daulet Amirkhanov
50ebcedf37 fix: natural_language_retriever.py 2025-08-15 10:54:11 +01:00
Daulet Amirkhanov
85c7f11eb8 Keep networkx as core dependency 2025-08-15 10:28:56 +01:00
misselvexu
81a17580a6 Remove NetworkX from cognee core. 2025-08-15 10:28:56 +01:00
hajdul88
d884cc46e9
Merge branch 'dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-15 08:15:05 +02:00
Igor Ilic
4543890a70
Loader separation (#1240)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: vasilije <vas.markovic@gmail.com>
2025-08-14 19:55:39 +02:00
hajdul88
dd6e26fe41 Merge remote-tracking branch 'origin/dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-14 16:17:36 +02:00
Daulet Amirkhanov
b297289060
Fix/add async lock to all vector databases (#1244)
<!-- .github/pull_request_template.md -->

## Description
1. Cleans up VectorDB adapters that have been migrated to
`cognee-community` repo
2. Adds async lock protection create_collection method in remaining
VectorDB - ChromaDB

See #1222

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
2025-08-14 15:57:34 +02:00
hajdul88
7af4775c56 chore: changed pypdf error base class 2025-08-13 15:17:01 +02:00
hajdul88
da40365932 ruff formatting 2025-08-13 15:15:39 +02:00
hajdul88
68327d3ab9 chore: Changing base classes for exceptions that were already defined 2025-08-13 15:14:43 +02:00
hajdul88
fabbd638a4 chore: renaming error 2025-08-13 15:03:12 +02:00
hajdul88
885f7c3f99 chore: fixing graph elements tests 2025-08-13 14:58:56 +02:00
hajdul88
38329da0e8
Merge branch 'dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-13 14:11:56 +02:00
Igor Ilic
beea2f5e0a
Incremental loading migration (#1238)
<!-- .github/pull_request_template.md -->

## Description
Add relational db migration for incremental loading, change incremental
loading to work document per document instead of async together

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-13 07:58:09 -04:00
hajdul88
6dcd59c73c feat: Adds changes to cognee graph part 2025-08-13 13:07:04 +02:00
hajdul88
9fb9f68c42 adds new base errors to retrieval exceptions 2025-08-13 12:36:31 +02:00
hajdul88
5bc00f1143 feat: adds new search classes to search.py 2025-08-13 12:29:35 +02:00
hajdul88
6870bba5a9 feat: adds new error to delete 2025-08-13 12:03:18 +02:00
hajdul88
7bd2660d08 feat: setting base classes of data exceptions to the new ones 2025-08-13 11:58:32 +02:00
EricXiao
815d639132
fix: graph visualization access for users with read permissions (#1220)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->
Description
This PR fix graph visualization access for users with read permissions
(https://github.com/topoteretes/cognee/issues/1182)

- Add permission checks for graph visualization endpoints to ensure
users can only access datasets they have permission to view
- Create get_dataset_with_permissions method to validate user access
before returning a dataset
- Remove redundant dataset existence validation in datasets router and
delegate permission checking to graph data retrieval
- Add comprehensive test suite for graph visualization permissions
covering owner access and permission granting scenarios
- Update get_formatted_graph_data() to use dataset owner's ID for
context
## Testing
Tests can be run with:
```bash
pytest -s cognee/tests/test_graph_visualization_permissions.py
```

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Signed-off-by: EricXiao <taoiaox@gmail.com>
Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2025-08-08 20:42:57 +02:00
lxobr
6dbd8e85a1
feat: dynamic multiple edges in datapoints (#1212)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->
- Improved list handling, removed `.index` logic from
`get_graph_from_model`, transitioned to fully datapoint-oriented
processing
- Streamlined datapoint iteration by introducing `_datapoints_generator`
with nested loops
- Generalized field processing to handle mixed lists: `[DataPoint,
(Edge, DataPoint), (Edge, [DataPoint])]`, allowing dynamic multiple
edges generation
- Small improvements and refactorings
- Added tests to `test_get_graph_from_model_flexible_edges()` covering
weighted edges and dynamic multiple edges
- Created `dynamic_multiple_edges_example.py` demonstrating dynamic
multiple edges

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-07 14:50:45 +02:00
Vasilije
dabd0912f8
feat: Cog 2082 add BAML to cognee (#1054)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Signed-off-by: Raj2604 <rajmandhare26@gmail.com>
Co-authored-by: Daulet Amirkhanov <damirkhanov01@gmail.com>
Co-authored-by: Hande <159312713+hande-k@users.noreply.github.com>
Co-authored-by: Igor Ilic <igorilic03@gmail.com>
Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
Co-authored-by: Boris <boris@topoteretes.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions@users.noreply.github.com>
Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
Co-authored-by: Raj Mandhare <96978537+Raj2604@users.noreply.github.com>
Co-authored-by: Pedro Thompson <thompsonp17@hotmail.com>
Co-authored-by: Pedro Henrique Thompson Furtado <pedrothompson@petrobras.com.br>
2025-08-06 10:41:47 +02:00
Igor Ilic
8d4ed35cbe
Fix low level pipeline (#1203)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-05 17:01:48 +02:00
EricXiao
fc7a91d991
feature: implement FEELING_LUCKY search type (#1178)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->
This PR implements the 'FEELING_LUCKY' search type, which intelligently
routes user queries to the most appropriate search retriever, addressing
[#1162](https://github.com/topoteretes/cognee/issues/1162).

- implement new search type FEELING_LUCKY
- Add the select_search_type function to analyze queries and choose the
proper search type
- Integrate with an LLM for intelligent search type determination
- Add logging for the search type selection process
- Support fallback to RAG_COMPLETION when the LLM selection fails
- Add tests for the new search type

## How it works
When a user selects the 'FEELING_LUCKY' search type, the system first
sends their natural language query to an LLM-based classifier. This
classifier analyzes the query's intent (e.g., is it asking for a
relationship, a summary, or a factual answer?) and selects the optimal
SearchType, such as 'INSIGHTS' or 'GRAPH_COMPLETION'. The main search
function then proceeds using this dynamically selected type. If the
classification process fails, it gracefully falls back to the default
'RAG_COMPLETION' type.

## Testing
Tests can be run with:
```bash
python -m pytest cognee/tests/unit/modules/search/search_methods_test.py -k "feeling_lucky" -v
```

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

Signed-off-by: EricXiao <taoiaox@gmail.com>
2025-08-02 16:30:08 +02:00
Igor Ilic
14ba3e8829
feat: Enable async execution of data items for incremental loading (#1092)
<!-- .github/pull_request_template.md -->

## Description
Attempt at making incremental loading run async

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-07-29 10:39:31 -04:00