Commit graph

2765 commits

Author SHA1 Message Date
hajdul88
c75f017eb9
Merge branch 'dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-14 10:53:31 +02:00
vasilije
0ed6f255c2 added fix 2025-08-14 10:52:39 +02:00
hajdul88
ca21122edb
Merge branch 'dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-14 10:09:40 +02:00
Pavan Chilukuri
1ea632d0fa
Add health checks (#1184)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

Replaces basic /health endpoint with production-ready health monitoring
system that checks all critical backend components (databases, storage,
LLM providers) for container orchestration and monitoring.

Changes
New: cognee/api/health.py - Core health check system with structured
monitoring

Enhanced: cognee/api/client.py - Three new health endpoints

Added: examples/health_check_example.py - Testing utilities

New Endpoints
GET /health - Liveness probe (HTTP 200/503)
<img width="480" height="351" alt="Screenshot 2025-08-12 at 10 15 45 AM"
src="https://github.com/user-attachments/assets/53279b53-51bf-45a1-b3b9-4546e7bb6730"
/>

GET /health/detailed - Complete component status with metrics
<img width="1012" height="244" alt="Screenshot 2025-08-12 at 10 17
12 AM"
src="https://github.com/user-attachments/assets/ad3ac9cd-1135-490f-9641-726c4ea4e724"
/>


Components Monitored
Critical (failure = 503): Relational DB, Vector DB, Graph DB, File
Storage
Non-critical (failure = degraded): LLM Provider, Embedding Service


## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: Daulet Amirkhanov <damirkhanov01@gmail.com>
Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2025-08-13 20:41:36 +02:00
hajdul88
dbb967fda8 chore: updates neptune exception base + ruff 2025-08-13 16:28:05 +02:00
hajdul88
748e9fad86 feat: adds s3 file system not found error to ingestion 2025-08-13 16:17:58 +02:00
hajdul88
d14d31adbe chore: updating delete_by_id test 2025-08-13 15:39:11 +02:00
hajdul88
8bd2416bc7 chore: fixing delete exception import in dataset router 2025-08-13 15:36:17 +02:00
hajdul88
66d2c75128 chore: changing docstring 2025-08-13 15:20:24 +02:00
hajdul88
7af4775c56 chore: changed pypdf error base class 2025-08-13 15:17:01 +02:00
hajdul88
da40365932 ruff formatting 2025-08-13 15:15:39 +02:00
hajdul88
68327d3ab9 chore: Changing base classes for exceptions that were already defined 2025-08-13 15:14:43 +02:00
hajdul88
59c9204a74 Update exceptions.py 2025-08-13 15:03:44 +02:00
hajdul88
fabbd638a4 chore: renaming error 2025-08-13 15:03:12 +02:00
hajdul88
d1bfeaa0f2 fix: fixes search unit test error expectation 2025-08-13 15:00:25 +02:00
hajdul88
885f7c3f99 chore: fixing graph elements tests 2025-08-13 14:58:56 +02:00
hajdul88
544e08930b feat: removing invalidValueErrors 2025-08-13 14:42:57 +02:00
hajdul88
38329da0e8
Merge branch 'dev' into feature/cog-2717-add-better-error-management-to-cognee 2025-08-13 14:11:56 +02:00
hajdul88
bed523a36b feat: api error handling restruct 2025-08-13 13:59:12 +02:00
Igor Ilic
beea2f5e0a
Incremental loading migration (#1238)
<!-- .github/pull_request_template.md -->

## Description
Add relational db migration for incremental loading, change incremental
loading to work document per document instead of async together

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-13 07:58:09 -04:00
hajdul88
32996aa0d0 feat: adds new error classes to llm and databases + introduces loglevel and logging from child error 2025-08-13 13:40:50 +02:00
hajdul88
6dcd59c73c feat: Adds changes to cognee graph part 2025-08-13 13:07:04 +02:00
hajdul88
91b9c11cd0 changes base class for vectordb exceptions 2025-08-13 12:39:34 +02:00
hajdul88
9fb9f68c42 adds new base errors to retrieval exceptions 2025-08-13 12:36:31 +02:00
hajdul88
5bc00f1143 feat: adds new search classes to search.py 2025-08-13 12:29:35 +02:00
hajdul88
657c775cbe feat: adds configattributeerror 2025-08-13 12:22:03 +02:00
hajdul88
ebd4403c2f adds new error classes to keywordextration error and defines new error 2025-08-13 12:14:42 +02:00
hajdul88
6870bba5a9 feat: adds new error to delete 2025-08-13 12:03:18 +02:00
hajdul88
7bd2660d08 feat: setting base classes of data exceptions to the new ones 2025-08-13 11:58:32 +02:00
hajdul88
623148ec6a feat: adds new errors to litellm client 2025-08-13 11:53:29 +02:00
hajdul88
b40dbf3c68 feat: adds new error classes to litellm instructor 2025-08-13 11:50:24 +02:00
hajdul88
1b3898dd8b feat: updates rel_db exceptions with the new error classes 2025-08-13 11:38:38 +02:00
hajdul88
0555dd9dda Update __init__.py 2025-08-13 11:29:16 +02:00
hajdul88
ed555a731d feat: adds new Exception classes 2025-08-13 11:27:45 +02:00
hajdul88
fd9aaf57b1
feature: Adds doctype handling to delete (audio, image, unstructured) (#1239)
<!-- .github/pull_request_template.md -->

## Description
feature: Adds doctype handling to delete (audio, image, unstructured)

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-12 19:01:06 +02:00
Vasilije
0c42d19505
Update README.md 2025-08-12 15:38:30 +02:00
Igor Ilic
a75a79f012
Lancedb async lock (#1222)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2025-08-12 08:46:15 -04:00
Igor Ilic
c33536685d
fix: Resolve Kuzu S3 issue (#1235)
<!-- .github/pull_request_template.md -->

## Description
Resolve issue with pushing Kuzu DBs to S3

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-11 16:29:36 +02:00
Hande
6e7e4506cc
chore: update readme (#1234)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-11 11:16:16 +02:00
EricXiao
815d639132
fix: graph visualization access for users with read permissions (#1220)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->
Description
This PR fix graph visualization access for users with read permissions
(https://github.com/topoteretes/cognee/issues/1182)

- Add permission checks for graph visualization endpoints to ensure
users can only access datasets they have permission to view
- Create get_dataset_with_permissions method to validate user access
before returning a dataset
- Remove redundant dataset existence validation in datasets router and
delegate permission checking to graph data retrieval
- Add comprehensive test suite for graph visualization permissions
covering owner access and permission granting scenarios
- Update get_formatted_graph_data() to use dataset owner's ID for
context
## Testing
Tests can be run with:
```bash
pytest -s cognee/tests/test_graph_visualization_permissions.py
```

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Signed-off-by: EricXiao <taoiaox@gmail.com>
Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2025-08-08 20:42:57 +02:00
Vasilije
e3b41e0ed4
fix: Interface fixes (#1206)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: Daulet Amirkhanov <damirkhanov01@gmail.com>
2025-08-08 20:41:33 +02:00
Vasilije
f65605b575
fix: Feature/cog 2648 evals update (#1221)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Co-authored-by: lxobr <122801072+lxobr@users.noreply.github.com>
Co-authored-by: Hande <159312713+hande-k@users.noreply.github.com>
2025-08-08 20:23:09 +02:00
lxobr
6dbd8e85a1
feat: dynamic multiple edges in datapoints (#1212)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->
- Improved list handling, removed `.index` logic from
`get_graph_from_model`, transitioned to fully datapoint-oriented
processing
- Streamlined datapoint iteration by introducing `_datapoints_generator`
with nested loops
- Generalized field processing to handle mixed lists: `[DataPoint,
(Edge, DataPoint), (Edge, [DataPoint])]`, allowing dynamic multiple
edges generation
- Small improvements and refactorings
- Added tests to `test_get_graph_from_model_flexible_edges()` covering
weighted edges and dynamic multiple edges
- Created `dynamic_multiple_edges_example.py` demonstrating dynamic
multiple edges

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-07 14:50:45 +02:00
vasilije
c8202c51a7 format fix 2025-08-06 19:11:39 +02:00
vasilije
0ea5894823 added distributed fixes 2025-08-06 19:11:20 +02:00
hajdul88
4e816ad80b
fix: changing deletion logic to use document id instead of content hash (#1210)
<!-- .github/pull_request_template.md -->

## Description
Changing deletion logic to use document id instead of content hash

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-06 16:54:35 +02:00
sssaha1989
a9e74dac42
Update LiteLLMEmbeddingEngine.py (#1205)
This will allow to deal with the issue when the user is using custom
embedding and LLM and passes the hosted_vllm option as part of the
LiteLLM documentation

<!-- .github/pull_request_template.md -->

## Description
<!-- This allows the user to use hosted_vllm with respect to LiteLLM
usage - and only gets applicable for custom embedding models -
specifically Hugging Face models -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-06 16:53:33 +02:00
Igor Ilic
b54e843951
Add neo4j multi db support (#1207)
<!-- .github/pull_request_template.md -->

## Description
Add multi db support for Neo4j Enterprise users

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Signed-off-by: Raj2604 <rajmandhare26@gmail.com>
Co-authored-by: vasilije <vas.markovic@gmail.com>
Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
Co-authored-by: Daulet Amirkhanov <damirkhanov01@gmail.com>
Co-authored-by: Hande <159312713+hande-k@users.noreply.github.com>
Co-authored-by: Boris <boris@topoteretes.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions@users.noreply.github.com>
Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
Co-authored-by: Raj Mandhare <96978537+Raj2604@users.noreply.github.com>
Co-authored-by: Pedro Thompson <thompsonp17@hotmail.com>
Co-authored-by: Pedro Henrique Thompson Furtado <pedrothompson@petrobras.com.br>
2025-08-06 10:44:45 +02:00
Vasilije
dabd0912f8
feat: Cog 2082 add BAML to cognee (#1054)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.

---------

Signed-off-by: Raj2604 <rajmandhare26@gmail.com>
Co-authored-by: Daulet Amirkhanov <damirkhanov01@gmail.com>
Co-authored-by: Hande <159312713+hande-k@users.noreply.github.com>
Co-authored-by: Igor Ilic <igorilic03@gmail.com>
Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
Co-authored-by: Boris <boris@topoteretes.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions@users.noreply.github.com>
Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
Co-authored-by: Raj Mandhare <96978537+Raj2604@users.noreply.github.com>
Co-authored-by: Pedro Thompson <thompsonp17@hotmail.com>
Co-authored-by: Pedro Henrique Thompson Furtado <pedrothompson@petrobras.com.br>
2025-08-06 10:41:47 +02:00
Igor Ilic
8d4ed35cbe
Fix low level pipeline (#1203)
<!-- .github/pull_request_template.md -->

## Description
<!-- Provide a clear description of the changes in this PR -->

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin.
2025-08-05 17:01:48 +02:00