<!-- .github/pull_request_template.md -->
## Description
<!-- Provide a clear description of the changes in this PR -->
## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced pipeline execution now provides consolidated status feedback
with improved telemetry for start, completion, and error events.
- Automatic generation of unique dataset identifiers offers clearer task
and pipeline run associations.
- **Refactor**
- Task execution has been streamlined with explicit parameter handling
for more structured pipeline processing.
- Interactive examples and demos now return results directly, making
integration and monitoring more accessible.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
<!-- .github/pull_request_template.md -->
## Description
Add test of embedding and LLM model at beginning of cognee use
Fix issue with relational database async use
Refactor handling of cache mechanism for all databases so changes in
config can be reflected in get functions
## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced connection testing for language and embedding services at
startup, ensuring improved reliability during data addition.
- **Refactor**
- Streamlined engine initialization across multiple database systems to
enhance performance and clarity.
- Improved parameter handling and caching strategies for faster, more
consistent operations.
- Updated record identifiers for more robust and unique data storage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: holchan <61059652+holchan@users.noreply.github.com>
Co-authored-by: Boris <boris@topoteretes.com>
* feat: Add support for LlamaIndex Document type
Added support for LlamaIndex Document type
Feature #COG-337
* docs: Add Jupyer Notebook for cognee with llama index document type
Added jupyter notebook which demonstrates cognee with LlamaIndex document type usage
Docs #COG-337
* feat: Add metadata migration from LlamaIndex document type
Allow usage of metadata from LlamaIndex documents
Feature #COG-337
* refactor: Change llama index migration function name
Change name of llama index function
Refactor #COG-337
* chore: Add llama index core dependency
Downgrade needed on tenacity and instructor modules to support llama index
Chore #COG-337
* Feature: Add ingest_data_with_metadata task
Added task that will have access to metadata if data is provided from different data ingestion tools
Feature #COG-337
* docs: Add description on why specific type checking is done
Explained why specific type checking is used instead of isinstance, as isinstace returns True for child classes as well
Docs #COG-337
* fix: Add missing parameter to function call
Added missing parameter to function call
Fix #COG-337
* refactor: Move storing of data from async to sync function
Moved data storing from async to sync
Refactor #COG-337
* refactor: Pretend ingest_data was changes instead of having two tasks
Refactor so ingest_data file was modified instead of having two ingest tasks
Refactor #COG-337
* refactor: Use old name for data ingestion with metadata
Merged new and old data ingestion tasks into one
Refactor #COG-337
* refactor: Return ingest_data and save_data_to_storage Tasks
Returned ingest_data and save_data_to_storage tasks
Refactor #COG-337
* refactor: Return previous ingestion Tasks to add function
Returned previous ignestion tasks to add function
Refactor #COG-337
* fix: Remove dict and use string for search query
Remove dictionary and use string for query in notebook and simple example
Fix COG-337
* refactor: Add changes request in pull request
Added the following changes that were requested in pull request:
Added synchronize label,
Made uniform syntax in if statement in workflow,
fixed instructor dependency,
added llama-index to be optional
Refactor COG-337
* fix: Resolve issue with llama-index being mandatory
Resolve issue with llama-index being mandatory to run cognee
Fix COG-337
* fix: Add install of llama-index to notebook
Removed additional references to llama-index from core cognee lib.
Added llama-index-core install from notebook
Fix COG-337
---------
Added first working iteration of PGVector for cognee, some important funcionality is still missing, but the core is there. Also some refactoring will be necessary.
Feature: #COG-170
