Commit graph

56 commits

Author SHA1 Message Date
Boris
348610e73c
fix: refactor get_graph_from_model to return nodes and edges correctly (#257)
* fix: handle rate limit error coming from llm model

* fix: fixes lost edges and nodes in get_graph_from_model

* fix: fixes database pruning issue in pgvector (#261)

* fix: cognee_demo notebook pipeline is not saving summaries

---------

Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-12-06 12:52:01 +01:00
Boris Arzentar
d49ab4c3b5 feat: update code-graph notebook 2024-12-03 23:48:12 +01:00
Vasilije
9d6081c7f7
feat: Add support for multiple audio and image formats (#12)
Added support for multiple audio and image formats with example

The formats added are the possible filetype library return values for
extension for Audio and Images

Feature COG-507
2024-11-23 16:31:55 +01:00
Igor Ilic
61ed516d12 docs: Add multimedia notebook
Added multimedia notebook for cognee

Docs COG-507
2024-11-20 16:21:29 +01:00
Igor Ilic
f9353d25fa fix: Update table name in notebook
Update table name in notebook

Fix COG-677
2024-11-20 15:14:38 +01:00
Igor Ilic
70fe6ac541 fix: Update table name in notebook
Update table name to use latest in notebook

Fix COG-677
2024-11-20 15:07:38 +01:00
Igor Ilic
4b55354dce
fix: Resolve issue with pgvector timeout (#3)
By creating PGVector as a singleton all issues regrading timeout are
resolved as there are no more parallel instances trying to communicate
with the database
2024-11-19 15:31:26 +01:00
Boris
5f144a0f92
fix: make all checks green (#1) 2024-11-19 15:30:09 +01:00
Boris
d8b6eeded5
feat: log search queries and results (#166)
* feat: log search queries and results

* fix: address coderabbit review comments

* fix: parse UUID when logging search results

* fix: remove custom UUID type and use DB agnostic UUID from sqlalchemy

* Add new cognee_db

---------

Co-authored-by: Leon Luithlen <leon@topoteretes.com>
2024-11-17 11:59:10 +01:00
Igor Ilic
d30adb53f3
Cog 337 llama index support (#186)
* feat: Add support for LlamaIndex Document type

Added support for LlamaIndex Document type

Feature #COG-337

* docs: Add Jupyer Notebook for cognee with llama index document type

Added jupyter notebook which demonstrates cognee with LlamaIndex document type usage

Docs #COG-337

* feat: Add metadata migration from LlamaIndex document type

Allow usage of metadata from LlamaIndex documents

Feature #COG-337

* refactor: Change llama index migration function name

Change name of llama index function

Refactor #COG-337

* chore: Add llama index core dependency

Downgrade needed on tenacity and instructor modules to support llama index

Chore #COG-337

* Feature: Add ingest_data_with_metadata task

Added task that will have access to metadata if data is provided from different data ingestion tools

Feature #COG-337

* docs: Add description on why specific type checking is done

Explained why specific type checking is used instead of isinstance, as isinstace returns True for child classes as well

Docs #COG-337

* fix: Add missing parameter to function call

Added missing parameter to function call

Fix #COG-337

* refactor: Move storing of data from async to sync function

Moved data storing from async to sync

Refactor #COG-337

* refactor: Pretend ingest_data was changes instead of having two tasks

Refactor so ingest_data file was modified instead of having two ingest tasks

Refactor #COG-337

* refactor: Use old name for data ingestion with metadata

Merged new and old data ingestion tasks into one

Refactor #COG-337

* refactor: Return ingest_data and save_data_to_storage Tasks

Returned ingest_data and save_data_to_storage tasks

Refactor #COG-337

* refactor: Return previous ingestion Tasks to add function

Returned previous ignestion tasks to add function

Refactor #COG-337

* fix: Remove dict and use string for search query

Remove dictionary and use string for query in notebook and simple example

Fix COG-337

* refactor: Add changes request in pull request

Added the following changes that were requested in pull request:

Added synchronize label,
Made uniform syntax in if statement in workflow,
fixed instructor dependency,
added llama-index to be optional

Refactor COG-337

* fix: Resolve issue with llama-index being mandatory

Resolve issue with llama-index being mandatory to run cognee

Fix COG-337

* fix: Add install of llama-index to notebook

Removed additional references to llama-index from core cognee lib.
Added llama-index-core install from notebook

Fix COG-337

---------
2024-11-17 11:47:08 +01:00
Boris Arzentar
7c015e525d fix: cognee_demo notebook search 2024-11-12 09:01:03 +01:00
Boris Arzentar
da4d9c2c3b fix: change entity collection name 2024-11-12 09:01:03 +01:00
Boris Arzentar
c0d1aa1216 fix: update entities collection name in cognee_demo notebook 2024-11-12 09:01:03 +01:00
Leon Luithlen
9fe1b6c5fa Add code_graph_demo notebook 2024-11-12 09:01:03 +01:00
Boris Arzentar
a2b1087c84 feat: add FalkorDB integration 2024-11-12 09:01:01 +01:00
Boris
52180eb6b5
feat: COG-184 add falkordb (#192)
* feat: add falkordb adapter

---------

Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-11-11 18:20:52 +01:00
Igor Ilic
23ed38d615 test: Fix intentional typo in notebook
Removed typo used for testing notebook github action

Test #COG-462
2024-10-29 14:24:17 +01:00
Igor Ilic
2ba57220d8 test: Add typo in notebook to test github action
Added typo in notebook to test if github action will catch the issue

Test #COG-462
2024-10-29 14:20:47 +01:00
Igor Ilic
c183742ad5 test: Add test for Jupyter notebook
Added testing of Jupyter notebook through github actions

Test #COG-462
2024-10-29 13:47:23 +01:00
Igor Ilic
6555f4e88e fix: Resolve chunking issue for notebook
Add cleaning of local data to resolve chunking issue with repeated notebook use

Fix
2024-10-27 22:33:20 +01:00
Boris
2f832b190c
fix: various fixes for the deployment
* fix: remove groups from UserRead model

* fix: add missing system dependencies for postgres

* fix: change vector db provider environment variable name

* fix: WeaviateAdapter retrieve bug

* fix: correctly return data point objects from retrieve method

* fix: align graph object properties

* feat: add node example
2024-10-22 11:26:48 +02:00
Igor Ilic
658b6df4c6 refactor: Remove architecture overview
Removed architecture overview from notebook for now

Refactor #COG-387
2024-10-11 17:57:51 +02:00
Igor Ilic
1446606e95 fix: Fix pruning of relation database for Notebook
Fixes issue regarding no pruning of relational database for Jupyter Notebook

Fix #COG-387
2024-10-10 18:03:27 +02:00
Igor Ilic
85bf48b1f8 refactor: Raise font of last header
Bigger font for last header regarding starring cognee

Refactor #COG-387
2024-10-10 17:15:20 +02:00
Igor Ilic
9a12e29c97 refactor: Add cognee process diagram to notebook
Added cognee process diagram for the data used in the notebook

Refactor #COG-387
2024-10-10 17:05:16 +02:00
Igor Ilic
2bfe185672 refactor: Expand cognee notebook introduction
Expand information about cognee in notebook

Refactor #COG-387
2024-10-10 15:39:13 +02:00
Igor Ilic
f308cdd6fe refactor: Refactor cognee text in notebook
Added cognee description from landing page with appropriate graphs and images

Refactor #COG-387
2024-10-10 15:27:48 +02:00
Igor Ilic
e5dd99d2ca refactor: refactor rag problem image
Refactor image to accomodate to cognee style

Refactor #COG-387
2024-10-10 13:14:24 +02:00
Igor Ilic
bd8f5284a9 refactor: Add other search types to notebook
Added other search types and explanations to notebook

Refactor #COG-387
2024-10-09 20:58:16 +02:00
Igor Ilic
9ec292aac5 refactor: Add other search types to notebook
Added other search types and explanations to jupyter notebook

refactor #COG-387
2024-10-09 17:52:29 +02:00
Igor Ilic
eab6bda2d3 refactor: Add summarization task to notebook
Added summarization task to jupyter notebook

Refactor #COG-387
2024-10-09 17:12:32 +02:00
Igor Ilic
2497546cfb refactor: Add link for articles
Added hyperlinks to article mention

Refactor #COG-382
2024-10-08 20:12:25 +02:00
Igor Ilic
85694687a3 refactor: Refactor RAG issue explanation
Refactored explanation about issues with RAGs. Added information on how results are scored.

Refactor #COG-382
2024-10-08 19:56:59 +02:00
Igor Ilic
b283896564 refactor: add LLMs problem description
Refactored most common problems with LLMs to notebook

Refactor #COG-382
2024-10-08 18:42:28 +02:00
Igor Ilic
a7e9fc8004 refactor: Add new photo of issues with RAGs
Added new diagram of the problems RAGs have I made with drawio

Refactor #COG-382
2024-10-08 18:11:02 +02:00
Igor Ilic
e80e36f3dc refactor: Change text size
Changed text size to be more readable and uniform across the notebook

Refactor #COG-382
2024-10-08 16:50:23 +02:00
Igor Ilic
e74f9ad45a refactor: Remove common questions from notebook
Removed segment regarding common questions from notebook

Refactor #COG-382
2024-10-08 13:29:05 +02:00
Igor Ilic
f2dfa647e0 fix: Add environment setup before pruning
Environment variables need to be setup properly before pruning

Fix #COG-382
2024-10-07 22:10:11 +02:00
Igor Ilic
a5ccf78f0f refactor: Refactor Jupyter Notebook
Combine all Jupyter Notebooks into one

Refactor #COG-382
2024-10-07 17:06:56 +02:00
Igor Ilic
fcd60861ba
fix: Fix Jupyter Notebook (#142)
* fix: resolve issue with dlt sqlalchemy usage
Cognee database configuration information was not handled properly by dlt, a new dlt handler
moudle was made to handle database configuration propagation.

* fix: resolve issue with jupyter notebook

cognee add function uses old way of working in the notebook, updated it to
work with the latest state of the cognee add function which doesn't return output.

* fix: Remove empty DB_PATH argument from .env.template

Empty value for DB_PATH in the .env file overrides default value for path intended to be used by cognee.

---------
2024-10-07 12:58:54 +02:00
Boris
58db1ac2c8
chore: increase the lib version (#138) 2024-09-21 17:57:35 +02:00
Boris
a9433e9283
feat: add sqlalchemy as dlt destination (#137)
* feat: add sqlalchemy as dlt destination

* Fix the demo, update Readme

* fix: add 1.5 notebook

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-09-21 15:58:28 +02:00
Boris
14555a25d0
feat: pipelines and tasks (#119)
* feat: simple graph pipeline

* feat: implement incremental graph generation

* fix: various bug fixes

* fix: upgrade weaviate-client

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-07-20 16:49:00 +02:00
Boris Arzentar
4fb3dc31a4 fix: enable sdk and fix config 2024-06-03 14:03:24 +02:00
Boris Arzentar
1ac28f4cb8 feat: add initial cognee frontend 2024-05-17 13:42:14 +02:00
Boris
219afbce68
feat: add lancedb vector storage [COG-176] (#90)
* feat: integrate lancedb

* fix: use futures in weaviate adapter to enable async behaviour
2024-05-03 10:35:41 +02:00
Boris Arzentar
9cbf450849 fix: move chunker to adapter 2024-04-23 14:59:09 +02:00
Boris Arzentar
e58251b00c fix: download nltk files when needed 2024-04-21 22:03:18 +02:00
Boris Arzentar
bdd664a2aa fix: replace dspy with official version and add custom dataset 2024-04-21 16:11:51 +02:00
Vasilije
bb679c2dd7
Improve processing, update networkx client, and Neo4j, and dspy (#69)
* Update cognify and the networkx client to prepare for running in Neo4j

* Fix for openai model

* Add the fix to the infra so that the models can be passed to the library. Enable llm_provider to be passed.

* Auto graph generation now works with neo4j

* Added fixes for both neo4j and networkx

* Explicitly name semantic node connections

* Added updated docs, readme, chunkers and updates to cognify

* Make docs build trigger only when changes on it happen

* Update docs, test git actions

* Separate cognify logic into tasks

* Introduce dspy knowledge graph extraction

---------
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
2024-04-20 19:05:40 +02:00