Commit graph

162 commits

Author SHA1 Message Date
Boris Arzentar
e07364fc25 Merge remote-tracking branch 'origin/main' into code-graph 2024-12-03 12:44:57 +01:00
Igor Ilic
6b97e95e14 refactor: Split entity related exceptions into graph and database exceptions
Move and split database entity related exceptions into graph and database exceptions

Refactor COG-502
2024-11-29 17:40:48 +01:00
Igor Ilic
df0b4b4820 refactor: Move user and group errors to users module
Moved user and group errors to users module

Refactor #COG-502
2024-11-29 17:06:34 +01:00
Leon Luithlen
7324564655 Add metadata_id attribute to Document and DocumentChunk, make ingest_with_metadata default 2024-11-28 09:18:49 +01:00
Igor Ilic
7d1210c889 feat: Add custom exception handling to dataset router
Added custom exceptions for dataset router

Feature COG-502
2024-11-27 15:57:37 +01:00
Igor Ilic
6eecc39db0 feat: Add custom exceptions to more cognee-lib modules
Added custom exceptions to more modules

Feature COG-502
2024-11-27 14:53:09 +01:00
Igor Ilic
ae568409a7 feat: Add custom exceptions to cognee lib
Added use of custom exceptions to cognee lib
2024-11-27 14:29:33 +01:00
Igor Ilic
a3fd079fd6 feat: Add Exceptions and exception handlers
Add classes for exceptions and add exception handling

Feature COG-502
2024-11-26 19:26:38 +01:00
Boris
d8b6eeded5
feat: log search queries and results (#166)
* feat: log search queries and results

* fix: address coderabbit review comments

* fix: parse UUID when logging search results

* fix: remove custom UUID type and use DB agnostic UUID from sqlalchemy

* Add new cognee_db

---------

Co-authored-by: Leon Luithlen <leon@topoteretes.com>
2024-11-17 11:59:10 +01:00
Igor Ilic
d30adb53f3
Cog 337 llama index support (#186)
* feat: Add support for LlamaIndex Document type

Added support for LlamaIndex Document type

Feature #COG-337

* docs: Add Jupyer Notebook for cognee with llama index document type

Added jupyter notebook which demonstrates cognee with LlamaIndex document type usage

Docs #COG-337

* feat: Add metadata migration from LlamaIndex document type

Allow usage of metadata from LlamaIndex documents

Feature #COG-337

* refactor: Change llama index migration function name

Change name of llama index function

Refactor #COG-337

* chore: Add llama index core dependency

Downgrade needed on tenacity and instructor modules to support llama index

Chore #COG-337

* Feature: Add ingest_data_with_metadata task

Added task that will have access to metadata if data is provided from different data ingestion tools

Feature #COG-337

* docs: Add description on why specific type checking is done

Explained why specific type checking is used instead of isinstance, as isinstace returns True for child classes as well

Docs #COG-337

* fix: Add missing parameter to function call

Added missing parameter to function call

Fix #COG-337

* refactor: Move storing of data from async to sync function

Moved data storing from async to sync

Refactor #COG-337

* refactor: Pretend ingest_data was changes instead of having two tasks

Refactor so ingest_data file was modified instead of having two ingest tasks

Refactor #COG-337

* refactor: Use old name for data ingestion with metadata

Merged new and old data ingestion tasks into one

Refactor #COG-337

* refactor: Return ingest_data and save_data_to_storage Tasks

Returned ingest_data and save_data_to_storage tasks

Refactor #COG-337

* refactor: Return previous ingestion Tasks to add function

Returned previous ignestion tasks to add function

Refactor #COG-337

* fix: Remove dict and use string for search query

Remove dictionary and use string for query in notebook and simple example

Fix COG-337

* refactor: Add changes request in pull request

Added the following changes that were requested in pull request:

Added synchronize label,
Made uniform syntax in if statement in workflow,
fixed instructor dependency,
added llama-index to be optional

Refactor COG-337

* fix: Resolve issue with llama-index being mandatory

Resolve issue with llama-index being mandatory to run cognee

Fix COG-337

* fix: Add install of llama-index to notebook

Removed additional references to llama-index from core cognee lib.
Added llama-index-core install from notebook

Fix COG-337

---------
2024-11-17 11:47:08 +01:00
Boris
52180eb6b5
feat: COG-184 add falkordb (#192)
* feat: add falkordb adapter

---------

Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-11-11 18:20:52 +01:00
Igor Ilic
ddf495266b Merge branch 'main' of github.com:topoteretes/cognee into COG-334-structure-routing 2024-11-05 22:53:33 +01:00
Igor Ilic
bce6540302 refactor: Add default value for datetime
Added None as default value for datetime

Refactor #COG-334
2024-11-05 22:40:14 +01:00
Igor Ilic
742540929c docs: Documentation update for getting and deleting Data
Updated docs to functions related to getting and deleting Data model objects

Docs #COG-334
2024-11-05 22:17:28 +01:00
Igor Ilic
801efeb1cb fix: Resolve security concerns regarding os calls
Resolved security concerns in endpoints regarding os

Fix #COG-334-structure-routing
2024-11-05 21:29:56 +01:00
Leon Luithlen
287427a240 Update litellm and openai version, bump pydantic 2024-11-04 15:45:06 +01:00
Igor Ilic
71ebfe7bb6 fix: Fix cognee graphistry and llm configuration through code
Fixed issue where graphistry and llm configuration options could not be set through code. Updated README.md

Fix #COG-472
2024-10-31 11:48:34 +01:00
Igor Ilic
eca1b9fddd feat: Add data deletion endpoint
Add ability to delete data instance through endpoint

Feature #COG-455
2024-10-28 14:40:49 +01:00
Igor Ilic
02d6750d05 refactor: Change class order in settings router
Changed class order in settings router to allow proper work of settings

Refactor #COG-334
2024-10-24 18:49:39 +02:00
Igor Ilic
4fd4651471 refactor: Add routers for datasets and add endpoints
Added routers for datasets and add endpoints

Refactor #COG-334
2024-10-24 17:56:15 +02:00
Igor Ilic
f8f2746603 refactor: Move permission router inside routers folder, add cognify router
Moved permissions router in it's routers folder, added cognify router

Refactor #COG-334
2024-10-24 17:18:50 +02:00
Igor Ilic
f09c28aada refactor: Add routers to search and settings
Added routers for search and settings endpoints

Refactor #COG-334
2024-10-24 17:01:34 +02:00
Igor Ilic
6b9a14270d refactor: Fix spacing, remove unused config methods
Remove unused config methods so we don't have to maintain them.
Fixed spacing in test_pgvector integration test.

Refactor #COG-170
2024-10-22 13:45:23 +02:00
Igor Ilic
88ded6e1a8 Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGvector-adapter 2024-10-22 12:41:18 +02:00
Igor Ilic
a3581689f2 refactor: Add setting of database configs through dictionary
Added the ability to set database configurations through dictionary for ease of use.
Updated test_pgvector to use this way of setting configuration

Refactor #COG-170
2024-10-22 11:53:33 +02:00
Boris
2f832b190c
fix: various fixes for the deployment
* fix: remove groups from UserRead model

* fix: add missing system dependencies for postgres

* fix: change vector db provider environment variable name

* fix: WeaviateAdapter retrieve bug

* fix: correctly return data point objects from retrieve method

* fix: align graph object properties

* feat: add node example
2024-10-22 11:26:48 +02:00
Igor Ilic
4a73505e23 refactor: Move create_db_and_tables module from vectors to pgvector
Moved create_db_and_tables module to pgvector so it only handles pgvector related db creation

Refactor #COG-170
2024-10-22 10:29:35 +02:00
Igor Ilic
05e4ef349e fix: Fix pruning of postgres database
Fixed issue with newly added tables not being pruned from postgres database

Fix #COG-170
2024-10-21 14:28:57 +02:00
Igor Ilic
58e5854943 Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGvector-adapter 2024-10-18 12:05:06 +02:00
Igor Ilic
9fbf2d857f feat: Add PGVector support
Added first working iteration of PGVector for cognee, some important funcionality is still missing, but the core is there. Also some refactoring will be necessary.

Feature: #COG-170
2024-10-17 17:05:38 +02:00
Boris
1eb4429c5c
feat: improve API request and response models and docs (#154)
* feat: improve API request and response models and docs
2024-10-14 13:38:36 +02:00
Igor Ilic
d68a3be32e feat: Add config support for pgvector
Added config support for using pgvector

Feature #COG-170
2024-10-11 13:23:11 +02:00
Boris
dc187a81d7
feat: migrate search to tasks (#144)
* fix: don't return anything on health endpoint

* feat: add alembic migrations

* feat: align search types with the data we store and migrate search to tasks
2024-10-07 14:41:35 +02:00
Igor Ilic
fcd60861ba
fix: Fix Jupyter Notebook (#142)
* fix: resolve issue with dlt sqlalchemy usage
Cognee database configuration information was not handled properly by dlt, a new dlt handler
moudle was made to handle database configuration propagation.

* fix: resolve issue with jupyter notebook

cognee add function uses old way of working in the notebook, updated it to
work with the latest state of the cognee add function which doesn't return output.

* fix: Remove empty DB_PATH argument from .env.template

Empty value for DB_PATH in the .env file overrides default value for path intended to be used by cognee.

---------
2024-10-07 12:58:54 +02:00
Boris Arzentar
279e4792a7 fix: await data and system prune 2024-10-01 19:43:17 +02:00
Boris Arzentar
3e88a314d8 fix: prune data and system before starting 2024-10-01 12:45:24 +02:00
Boris Arzentar
c257bc7d87 fix: remove default user creation on start 2024-10-01 12:37:31 +02:00
Boris
01582d7a55
feat: split add into tasks and use pipeline architecture (#141)
* feat: split add into tasks and use pipeline architecture
2024-09-30 14:09:20 +02:00
Boris
56868d8a6f
feat: add telemetry logging to pipelines and tasks (#140)
* feat: add telemetry logging to pipelines and tasks

* fix: enable telemetry for local environment
2024-09-29 12:20:48 +02:00
Boris Arzentar
41bf8617f1 fix: prune the system on startup 2024-09-25 14:59:04 +02:00
Boris Arzentar
d39d859e2b chore: enable all origins in cors settings 2024-09-25 14:34:14 +02:00
Boris
a9433e9283
feat: add sqlalchemy as dlt destination (#137)
* feat: add sqlalchemy as dlt destination

* Fix the demo, update Readme

* fix: add 1.5 notebook

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-09-21 15:58:28 +02:00
Boris
e1a0b55a21
feat: user authentication in routes (#133)
* feat: require logged in user in routes
2024-09-08 21:12:49 +02:00
Vasilije
22c0dd5b2d
Rewrite cognee documentation and apply theme (#130)
* Update docs

* fix: add cognee colors and logo

* fix: add link to community discord

---------

Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
2024-08-22 13:38:16 +02:00
Boris
94a674a088
feat: split document reader from chunker (#131)
* fix: abstract chunking into a separate class

* fix: yield merged text from text chunker

* fix: split python version tests

* fix: change postgres live check

* fix: remove unnecessary code

* fix: update checkout action

* fix: update setup-python action

* fix: add PG_USER env variable

* fix: make sure relationship_name is used everywhere

* fix: remove duplicate import
2024-08-19 14:36:10 +02:00
Vasilije
44b9e2b442 Enable different chunking methods 2024-08-08 20:34:20 +02:00
Vasilije
e80d39167b Enable different chunking methods 2024-08-08 19:59:26 +02:00
Vasilije
64bec1f1a4 Refactor of the tasks 2024-08-08 17:11:28 +02:00
Vasilije
4675a8f323 Refactor of the tasks 2024-08-08 17:10:43 +02:00
Vasilije
1c9bbd7a43 Refactor of the tasks 2024-08-08 17:08:19 +02:00