<!-- .github/pull_request_template.md -->
## Description
<!-- Provide a clear description of the changes in this PR -->
## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **Tests**
- Added comprehensive unit tests for graph model generation
- Introduced new test scenarios covering various data structures and
edge cases
- Implemented tests for document, chunk, and entity relationships
- **Chores**
- Updated continuous deployment workflow to trigger only on `dev` branch
The release focuses on improving test coverage and refining the
deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- .github/pull_request_template.md -->
## Description
PR to test Gemini PR from holchan
1. Add Gemini LLM and Gemini Embedding support
2. Fix CodeGraph issue with chunks being bigger than maximum token value
3. Add Tokenizer adapters to CodeGraph
## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for the Gemini LLM provider.
- Expanded LLM configuration options.
- Introduced a new GitHub Actions workflow for multimetric QA
evaluation.
- Added new environment variables for LLM and embedding configurations
across various workflows.
- **Bug Fixes**
- Improved error handling in various components.
- Updated tokenization and embedding processes.
- Removed warning related to missing `dict` method in data items.
- **Refactor**
- Simplified token extraction and decoding methods.
- Updated tokenizer interfaces.
- Removed deprecated dependencies.
- Enhanced retry logic and error handling in embedding processes.
- **Documentation**
- Updated configuration comments and settings.
- **Chores**
- Updated GitHub Actions workflows to accommodate new secrets and
environment variables.
- Modified evaluation parameters.
- Adjusted dependency management for optional libraries.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: holchan <61059652+holchan@users.noreply.github.com>
Co-authored-by: Boris <boris@topoteretes.com>
* feat: adds some unit tests for get_graph_from_model
* feat: updates neo4j add_edges cypher and deletes shallow get_graph_from_model
* fix: fixing merge conflict false resolve
* chore: deletes old only_root unit test
* Count the number of tokens in documents
* save token count to relational db
* Add metrics to metric table
* Store list as json instead of array in relational db table
* Sum in sql instead of python
* Unify naming
* Return data_points in descriptive metric calculation task
---------
Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
* Count the number of tokens in documents
* save token count to relational db
---------
Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
* feat: make tasks a configurable argument in the cognify function
* fix: add data points task
* Define pydantic models for descriptive graph metrics and input metrics
* remove to_json method
* Use just one MetricData class instead of two
---------
Co-authored-by: lxobr <122801072+lxobr@users.noreply.github.com>
* feat: make tasks a configurable argument in the cognify function
* fix: add data points task
* Ugly hack for multi-metric eval bug
* some cleanup
---------
Co-authored-by: lxobr <122801072+lxobr@users.noreply.github.com>
