Commit graph

293 commits

Author SHA1 Message Date
Igor Ilic
b60f2603f4 test: Add test for pgvector to confirm database deletion is working
Added assert to verify all tables in database have been cleared.
Added method to SqlAlchemyAdapter to get all table names in database.

Test COG-488
2024-11-20 17:11:23 +01:00
hajdul88
0101d43b8d feat: Adds graph node filtering by feature 2024-11-20 15:13:38 +01:00
hajdul88
44ac9b68b4 feat: adds get_distances from collection method to LanceDB and PgVector 2024-11-19 16:39:45 +01:00
Igor Ilic
4b55354dce
fix: Resolve issue with pgvector timeout (#3)
By creating PGVector as a singleton all issues regrading timeout are
resolved as there are no more parallel instances trying to communicate
with the database
2024-11-19 15:31:26 +01:00
Boris
d8b6eeded5
feat: log search queries and results (#166)
* feat: log search queries and results

* fix: address coderabbit review comments

* fix: parse UUID when logging search results

* fix: remove custom UUID type and use DB agnostic UUID from sqlalchemy

* Add new cognee_db

---------

Co-authored-by: Leon Luithlen <leon@topoteretes.com>
2024-11-17 11:59:10 +01:00
hajdul88
867e18de86 fix: Changes GraphDBInterface typing in CogneeGraph 2024-11-14 14:01:20 +01:00
hajdul88
d3fdddaa52 Revert "Checks the pgvector test issue"
This reverts commit 0d27371467.
2024-11-13 17:55:52 +01:00
hajdul88
0d27371467 Checks the pgvector test issue 2024-11-13 17:51:25 +01:00
hajdul88
9b62617394
Merge branch 'main' into feat/COG-553-graph-memory-projection 2024-11-12 16:56:40 +01:00
Vasilije
be792a7ba6
Merge branch 'main' into COG-575-remove-graph-overwrite-on-error 2024-11-12 10:18:09 +01:00
hajdul88
e988a67466 Fixes LanceDB datapoint add 2024-11-11 19:28:17 +01:00
Boris
52180eb6b5
feat: COG-184 add falkordb (#192)
* feat: add falkordb adapter

---------

Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-11-11 18:20:52 +01:00
Leon Luithlen
058b5916a4 Remove graph overwriting on exception in NetworkXAdapter 2024-11-11 12:59:36 +01:00
Igor Ilic
ddf495266b Merge branch 'main' of github.com:topoteretes/cognee into COG-334-structure-routing 2024-11-05 22:53:33 +01:00
Leon Luithlen
115e93b791 Remove fastembed dependency 2024-11-04 17:02:47 +01:00
Igor Ilic
d5a220ef0b test: Test telemetry
Removed space to trigger github actions to test telemetry

Test
2024-11-02 16:49:56 +01:00
Igor Ilic
de1ba5cd7c feat: Add cascade deletion for datasets and data
Added cascade deletion so when a dataset or data is deleted the connection in the dataset_data table is also deleted

Feature #COG-455
2024-11-01 12:55:20 +01:00
Igor Ilic
3567e0d7e7 fix: Fix chunk naive llm classifier
Fixed chunk naive llm classifier uuid issue, added fix for deletion of data points for LanceDB

Fix #COG-472
2024-10-31 00:42:18 +01:00
Igor Ilic
c23a314b63
COG-414: fix postgres database deletion (#163)
* fix: Add deletion of all tables in all schemas for postgres

Added deletion of all tables in postgres database, but this fix causes an issue regrading creation of duplicate tables on next run

Fix #COG-414

* fix: Resolve issue with database deletion

Resolve issue with database deletion by cleaning Metadata after every schema

Fix #COG-414

* fix: Move cleaning of MetaData out of drop table loop

Moved cleaning of MetaData to be after all tables have been dropped

Fix #COG-414

* refactor: Remove unnecessary print statement

Removed unnecessary print statement

Refactor #COG-414

* fix: Fix table deletion for SqlAlchemyAdapter

Fixed deletion of tables in SqlAlchemyAdapter so it works for sqlite and postgres

Fix #COG-414

* feat: Add deletion by id for SqlAlchemyAdapter

Added ability to delete data from database by id

Feature #COG-414

* fix: Add support for postgresql syntax for getting table function in SqlAlchemyAdapter

Added support for schema namespace for getting tables

Fix #COG-414
2024-10-28 09:57:30 +01:00
Igor Ilic
1088b58f11
fix: Add missing await to create_database (#162)
Added missing await for database creation in PGVectorAdapter

Fix #COG-170
2024-10-22 20:15:36 +02:00
Igor Ilic
0c6f019838 refactor: Remove broad exception handling from PGVectorAdapter
Removed broad exception handling for the search function in PGVectorAdapter

Refactor #COG-170
2024-10-22 15:41:07 +02:00
Igor Ilic
dc46304a8d fix: Add missing await statement to LanceDBAdapter and PGVectorAdapter
Added missing await statement to batch search for LanceDB and PGVector adapters

Fix #COG-170
2024-10-22 15:15:45 +02:00
Igor Ilic
d30c337370 refactor: Use SQLAlchemyAdapter create_database
Added reusing of sqlalchemy adapters create_database method instead of directly writing code for database creation

Refactor #COG-170
2024-10-22 14:23:36 +02:00
Igor Ilic
c7ed46ddaa fix: Change to new syntax for vector_engine_provider
Syntax changed for vector_engine_provider to vector_db_provider

Fix #COG-170
2024-10-22 12:57:17 +02:00
Igor Ilic
88ded6e1a8 Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGvector-adapter 2024-10-22 12:41:18 +02:00
Boris
2f832b190c
fix: various fixes for the deployment
* fix: remove groups from UserRead model

* fix: add missing system dependencies for postgres

* fix: change vector db provider environment variable name

* fix: WeaviateAdapter retrieve bug

* fix: correctly return data point objects from retrieve method

* fix: align graph object properties

* feat: add node example
2024-10-22 11:26:48 +02:00
Igor Ilic
4a73505e23 refactor: Move create_db_and_tables module from vectors to pgvector
Moved create_db_and_tables module to pgvector so it only handles pgvector related db creation

Refactor #COG-170
2024-10-22 10:29:35 +02:00
Igor Ilic
71c1374859 refactor: Move serialize_datetime function
Move serialize_datetime function to be in it's own file

Refactor #COG-170
2024-10-22 10:19:39 +02:00
Igor Ilic
05e4ef349e fix: Fix pruning of postgres database
Fixed issue with newly added tables not being pruned from postgres database

Fix #COG-170
2024-10-21 14:28:57 +02:00
Igor Ilic
240c660eac refactor: Change raw SQL queries to SQLalchemy ORM for PGVectorAdapter
Changed raw SQL quries to use SQLalchemy ORM for PGVectorAdapter

Refactor #COG-170
2024-10-21 12:59:24 +02:00
Igor Ilic
d2772d22b8 refactor: Formatting change for create_vector_engine
Change formatting for connection string for PGVectorAdapter

Refactor #COG-170
2024-10-18 15:03:28 +02:00
Igor Ilic
7f7b015ea3 refactor: Add formatting to create_vector_engine
Added formatting for PGVector part of create_vector_engine

Refactor #COG-170
2024-10-18 15:00:51 +02:00
Igor Ilic
2cd255768e refactor: Add formatting to PGVector Adapter
Formatted PGVectorAdapter

Refactor #COG-170
2024-10-18 14:46:33 +02:00
Igor Ilic
325e6cd654 refactor: Rewrite search query
Optimize search query for PGVector

Refactor #COG-170
2024-10-18 13:33:40 +02:00
Igor Ilic
02cd2408d6 feat: Add batch search to PGVectorAdapter
Added batch search to PGVectorAdapter

Feature #COG-170
2024-10-17 18:11:11 +02:00
Igor Ilic
aa26eabdbb refactor: Remove echo for database
Removed echoing of database operations

Refactor #COG-170
2024-10-17 17:26:26 +02:00
Igor Ilic
9b9ae6c8aa refactor: Remove unused env parameter
Remove VECTOR_DB_NAME env parameter as it's not needed

Refactor #COG-170
2024-10-17 17:13:40 +02:00
Igor Ilic
9fbf2d857f feat: Add PGVector support
Added first working iteration of PGVector for cognee, some important funcionality is still missing, but the core is there. Also some refactoring will be necessary.

Feature: #COG-170
2024-10-17 17:05:38 +02:00
Igor Ilic
268396abdc feature: Checkpoint during pgvector integration development
Saving state of pgvector integration development so far

Feature #COG-170
2024-10-11 17:11:05 +02:00
Igor Ilic
c62dfdda9b feat: Add PGVectorAdapter
Added PGVectorAdapter

Feature #COG-170
2024-10-11 15:00:28 +02:00
Igor Ilic
d68a3be32e feat: Add config support for pgvector
Added config support for using pgvector

Feature #COG-170
2024-10-11 13:23:11 +02:00
Boris
dc187a81d7
feat: migrate search to tasks (#144)
* fix: don't return anything on health endpoint

* feat: add alembic migrations

* feat: align search types with the data we store and migrate search to tasks
2024-10-07 14:41:35 +02:00
Igor Ilic
fcd60861ba
fix: Fix Jupyter Notebook (#142)
* fix: resolve issue with dlt sqlalchemy usage
Cognee database configuration information was not handled properly by dlt, a new dlt handler
moudle was made to handle database configuration propagation.

* fix: resolve issue with jupyter notebook

cognee add function uses old way of working in the notebook, updated it to
work with the latest state of the cognee add function which doesn't return output.

* fix: Remove empty DB_PATH argument from .env.template

Empty value for DB_PATH in the .env file overrides default value for path intended to be used by cognee.

---------
2024-10-07 12:58:54 +02:00
Boris Arzentar
2e70c23de8 fix: log db path 2024-10-01 20:39:22 +02:00
Boris
a9433e9283
feat: add sqlalchemy as dlt destination (#137)
* feat: add sqlalchemy as dlt destination

* Fix the demo, update Readme

* fix: add 1.5 notebook

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-09-21 15:58:28 +02:00
Boris
e1a0b55a21
feat: user authentication in routes (#133)
* feat: require logged in user in routes
2024-09-08 21:12:49 +02:00
Boris
94a674a088
feat: split document reader from chunker (#131)
* fix: abstract chunking into a separate class

* fix: yield merged text from text chunker

* fix: split python version tests

* fix: change postgres live check

* fix: remove unnecessary code

* fix: update checkout action

* fix: update setup-python action

* fix: add PG_USER env variable

* fix: make sure relationship_name is used everywhere

* fix: remove duplicate import
2024-08-19 14:36:10 +02:00
Vasilije
1c9bbd7a43 Refactor of the tasks 2024-08-08 17:08:19 +02:00
Vasilije
2e367198cd Task updates and updates to SQLAlchemy Adapter 2024-08-07 18:21:14 +02:00
Vasilije
a782286b93 Task updates and updates to SQLAlchemy Adapter 2024-08-07 12:36:02 +02:00