Commit graph

71 commits

Author SHA1 Message Date
Igor Ilic
764c0895df fix: Resolve Milvus connection issue, add config to milvus test, add milvus gh action
Resolved if statement resolution issue regrading api key,
Added vector db config to milvus test,
Added milvus gh action

Fix
2024-12-03 11:13:54 +01:00
Ryan Lin
f65070087f Feature: Integrate Milvus as the VectorDatabase 2024-12-03 03:40:28 -05:00
Igor Ilic
dcb320da52 chore: Add groq and langfuse as optional dependencies
Added groq and langfuse as optional dependencies

Chore COG-595
2024-12-02 19:35:50 +01:00
Igor Ilic
ad56ff9c1a chore: Add langchain as optional dependency
Added langchain as optional dependency

Chore COG-595
2024-12-02 16:47:26 +01:00
Igor Ilic
63e687978e chore: Remove falkordb and posthog non optional dependency
Switched falkordb and posthog to be optional dependencies

Chore COG-595
2024-12-02 15:30:30 +01:00
Igor Ilic
10dc6b1524 chore: Add optional dependencies
Change intended optional dependencies to really be optional

Chore COG-595
2024-12-02 14:38:41 +01:00
Igor Ilic
4b55354dce
fix: Resolve issue with pgvector timeout (#3)
By creating PGVector as a singleton all issues regrading timeout are
resolved as there are no more parallel instances trying to communicate
with the database
2024-11-19 15:31:26 +01:00
Igor Ilic
d30adb53f3
Cog 337 llama index support (#186)
* feat: Add support for LlamaIndex Document type

Added support for LlamaIndex Document type

Feature #COG-337

* docs: Add Jupyer Notebook for cognee with llama index document type

Added jupyter notebook which demonstrates cognee with LlamaIndex document type usage

Docs #COG-337

* feat: Add metadata migration from LlamaIndex document type

Allow usage of metadata from LlamaIndex documents

Feature #COG-337

* refactor: Change llama index migration function name

Change name of llama index function

Refactor #COG-337

* chore: Add llama index core dependency

Downgrade needed on tenacity and instructor modules to support llama index

Chore #COG-337

* Feature: Add ingest_data_with_metadata task

Added task that will have access to metadata if data is provided from different data ingestion tools

Feature #COG-337

* docs: Add description on why specific type checking is done

Explained why specific type checking is used instead of isinstance, as isinstace returns True for child classes as well

Docs #COG-337

* fix: Add missing parameter to function call

Added missing parameter to function call

Fix #COG-337

* refactor: Move storing of data from async to sync function

Moved data storing from async to sync

Refactor #COG-337

* refactor: Pretend ingest_data was changes instead of having two tasks

Refactor so ingest_data file was modified instead of having two ingest tasks

Refactor #COG-337

* refactor: Use old name for data ingestion with metadata

Merged new and old data ingestion tasks into one

Refactor #COG-337

* refactor: Return ingest_data and save_data_to_storage Tasks

Returned ingest_data and save_data_to_storage tasks

Refactor #COG-337

* refactor: Return previous ingestion Tasks to add function

Returned previous ignestion tasks to add function

Refactor #COG-337

* fix: Remove dict and use string for search query

Remove dictionary and use string for query in notebook and simple example

Fix COG-337

* refactor: Add changes request in pull request

Added the following changes that were requested in pull request:

Added synchronize label,
Made uniform syntax in if statement in workflow,
fixed instructor dependency,
added llama-index to be optional

Refactor COG-337

* fix: Resolve issue with llama-index being mandatory

Resolve issue with llama-index being mandatory to run cognee

Fix COG-337

* fix: Add install of llama-index to notebook

Removed additional references to llama-index from core cognee lib.
Added llama-index-core install from notebook

Fix COG-337

---------
2024-11-17 11:47:08 +01:00
Leon Luithlen
0c5ac33aff Adjust poetry.lock 2024-11-14 15:54:28 +01:00
Boris
52180eb6b5
feat: COG-184 add falkordb (#192)
* feat: add falkordb adapter

---------

Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-11-11 18:20:52 +01:00
Leon Luithlen
115e93b791 Remove fastembed dependency 2024-11-04 17:02:47 +01:00
Leon Luithlen
287427a240 Update litellm and openai version, bump pydantic 2024-11-04 15:45:06 +01:00
Boris
351355a48d
feature: add tracking to docs website (#165)
* feat: add segment tracking snippet
2024-10-25 14:09:27 +02:00
Igor Ilic
c78627f926 chore: Remove postgres group from pyproject.toml install postgres dependencies through extras
To be uniform with how we install optional dependencies a change has been made so that postgres optional dependencies are installed through extras

Chore #COG-170
2024-10-22 14:02:36 +02:00
Igor Ilic
dbc86e284a chore: Add pgvector back to mandatory dependencies
pgvector is imported directly in PGVector adapter therefore it needs to be installed with mandatory dependencies

Chore #COG-170
2024-10-22 12:51:26 +02:00
Igor Ilic
88ded6e1a8 Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGvector-adapter 2024-10-22 12:41:18 +02:00
Igor Ilic
7b2022e5f3 refactor: Move psycopg2 to an optional dependency
Moved psycopg2 as an optional dependency for postgres to not block users with lib issues it might cause

Refactor #COG-170
2024-10-22 12:24:51 +02:00
Boris
2f832b190c
fix: various fixes for the deployment
* fix: remove groups from UserRead model

* fix: add missing system dependencies for postgres

* fix: change vector db provider environment variable name

* fix: WeaviateAdapter retrieve bug

* fix: correctly return data point objects from retrieve method

* fix: align graph object properties

* feat: add node example
2024-10-22 11:26:48 +02:00
Igor Ilic
9461ba0fd1 chore: Add psycopg2 dependency
Add psycopg2 a postgresql database adapter dependency

Chore #COG-170
2024-10-21 15:32:51 +02:00
Igor Ilic
4c381a3ae9 chore: Add pgvector dependency
Added pgvector dependency for PGVectorAdapter

Chore #COG-170
2024-10-21 15:08:27 +02:00
Boris
dc187a81d7
feat: migrate search to tasks (#144)
* fix: don't return anything on health endpoint

* feat: add alembic migrations

* feat: align search types with the data we store and migrate search to tasks
2024-10-07 14:41:35 +02:00
Boris
a9433e9283
feat: add sqlalchemy as dlt destination (#137)
* feat: add sqlalchemy as dlt destination

* Fix the demo, update Readme

* fix: add 1.5 notebook

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-09-21 15:58:28 +02:00
Vasilije
a782286b93 Task updates and updates to SQLAlchemy Adapter 2024-08-07 12:36:02 +02:00
Boris Arzentar
a34acbc0ce fix: update neo4j lib 2024-08-06 16:22:17 +02:00
Boris Arzentar
3e3134b19f fix: fix debugpy version 2024-08-06 15:45:40 +02:00
Vasilije
07e2bc1b39 Fixes to the pipeline 2024-08-05 09:56:45 +02:00
Boris Arzentar
401167bb0f fix: enable sqlalchemy adapter 2024-08-04 22:23:28 +02:00
Vasilije
14e1eba363 Fixes for user flow with group management 2024-07-23 13:43:08 +02:00
Vasilije
e785b30117 Initial functional user auth 2024-07-22 23:59:16 +02:00
Boris Arzentar
6ef4bbe862 chore: remove unused dependencies 2024-07-20 16:59:25 +02:00
Boris
14555a25d0
feat: pipelines and tasks (#119)
* feat: simple graph pipeline

* feat: implement incremental graph generation

* fix: various bug fixes

* fix: upgrade weaviate-client

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-07-20 16:49:00 +02:00
Boris Arzentar
eac6ffac1a fix: remove dspy dependency 2024-06-30 12:25:30 +02:00
Boris Arzentar
cb64ab14fa chore: add vanilla docker config 2024-06-23 00:36:34 +02:00
Boris Arzentar
7c66364899 test: add weaviate integration test 2024-06-12 22:32:13 +02:00
Vasilije
11231b7ada rewrote chunking config 2024-06-09 22:46:02 +02:00
Vasilije
5ef81956c9 FIXES FOR COLLAB 2024-05-27 14:31:42 +02:00
Vasilije
8f157ed72e implement settings 2024-05-25 18:21:19 +02:00
Boris Arzentar
84c0c8cab5 feat: add llm config 2024-05-22 22:36:30 +02:00
Vasilije
aa2d6e97f0 fix docker compose 2024-05-18 08:12:04 +02:00
Boris Arzentar
1ac28f4cb8 feat: add initial cognee frontend 2024-05-17 13:42:14 +02:00
Boris
219afbce68
feat: add lancedb vector storage [COG-176] (#90)
* feat: integrate lancedb

* fix: use futures in weaviate adapter to enable async behaviour
2024-05-03 10:35:41 +02:00
Vasilije
212e5dcf78
Cog 174 (#84)
* Add telemetry

* test: add github action test

* fix: create graph only once

* fix: handle graph file not existing while deleting it

* fix: close qdrant connection in methods

---------

Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
2024-04-26 00:16:03 +02:00
Vasilije
1c2c72b8bf
Add telemetry (#82)
* Add telemetry
2024-04-24 19:06:53 +02:00
Boris Arzentar
bdd664a2aa fix: replace dspy with official version and add custom dataset 2024-04-21 16:11:51 +02:00
Vasilije
bb679c2dd7
Improve processing, update networkx client, and Neo4j, and dspy (#69)
* Update cognify and the networkx client to prepare for running in Neo4j

* Fix for openai model

* Add the fix to the infra so that the models can be passed to the library. Enable llm_provider to be passed.

* Auto graph generation now works with neo4j

* Added fixes for both neo4j and networkx

* Explicitly name semantic node connections

* Added updated docs, readme, chunkers and updates to cognify

* Make docs build trigger only when changes on it happen

* Update docs, test git actions

* Separate cognify logic into tasks

* Introduce dspy knowledge graph extraction

---------
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
2024-04-20 19:05:40 +02:00
Boris Arzentar
370b74988e chore: add functions to improve user experience 2024-03-30 15:25:34 +01:00
Boris Arzentar
fc7851156a chore: remove unused libs 2024-03-29 14:36:58 +01:00
Boris Arzentar
cf39bb1c85 chore: update python version to 3.11 2024-03-29 14:10:20 +01:00
Vasilije
b950dda3f6 Updated instructor 2024-03-29 11:10:37 +01:00
Vasilije
90c41512ed Ollama fixes, missing libs + config fixes 2024-03-28 11:26:22 +01:00