Commit graph

71 commits

Author SHA1 Message Date
Preston Rasmussen
38dd3e8dc3
Edge search updates (#753)
* update edge fulltext search

* update

* update
2025-07-22 10:05:58 -04:00
Preston Rasmussen
e56ba1a71c
save edge update (#721) 2025-07-14 11:15:38 -04:00
Daniel Chalef
4481702c9f
feat: enhance GeminiClient with max tokens management (#712)
* feat: enhance GeminiClient with max tokens management

- Introduced a mapping for maximum output tokens for various Gemini models.
- Added methods to resolve max tokens based on precedence rules, allowing for more flexible token management.
- Updated tests to verify max tokens behavior, ensuring explicit parameters take precedence and fallback mechanisms work correctly.

This change improves the handling of token limits for different models, enhancing the client’s configurability and usability.

* refactor: streamline max tokens retrieval in GeminiClient

- Removed the fallback to DEFAULT_MAX_TOKENS in favor of directly using model-specific maximum tokens.
- Simplified the logic for determining max tokens, enhancing code clarity and maintainability.

This change improves the efficiency of token management within the GeminiClient.
2025-07-13 14:37:55 -07:00
alan blount
e16740be9d
feat(gemini): embedding batch size & lite default (#680)
* feat(gemini): embedding batch size & lite default

The new `gemini-embedding-001` model only allows one embedding input per batch
(instance), but has other impressive statistics:
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api

The -DEFAULT_SMALL_MODEL must not have the 'models/' prefix.

* Refactor: Improve Gemini Client Error Handling and Reliability

This commit introduces several improvements to the Gemini client to enhance its robustness and reliability.

- Implemented more specific error handling for various Gemini API responses, including rate limits and safety blocks.
- Added a JSON salvaging mechanism to gracefully handle incomplete or malformed JSON responses from the API.
- Introduced detailed logging for failed LLM generations to simplify debugging and troubleshooting.
- Refined the Gemini embedder to better handle empty or invalid embedding responses.
- Updated and corrected tests to align with the improved error handling and reliability features.

* fix: cleanup in _log_failed_generation()

* fix: cleanup in _log_failed_generation()

* Fix ruff B904 error in gemini_client.py

* fix(gemini): correct retry logic and enhance error logging

Updated the retry mechanism in the GeminiClient to ensure it retries the maximum number of times specified. Improved error logging to provide clearer insights when all retries are exhausted, including detailed information about the last error encountered.

* fix(gemini): enhance error handling for safety blocks and update tests

Refined error handling in the GeminiClient to improve detection of safety block conditions. Updated test cases to reflect changes in exception messages and ensure proper retry logic is enforced. Enhanced mock responses in tests to better simulate real-world scenarios, including handling of invalid JSON responses.

* revert default gemini to text-embedding-001

---------

Co-authored-by: Daniel Chalef <131175+danielchalef@users.noreply.github.com>
2025-07-13 10:20:22 -07:00
Daniel Chalef
aa6e38856a
[REFACTOR][FIX] Move away from DEFAULT_DATABASE environment variable in favour of driver-config support (dc) (#699)
* fix: remove global DEFAULT_DATABASE usage in favor of driver-specific
config

Fixes bugs introduced in PR #607. This removes reliance on the global
DEFAULT_DATABASE environment variable. It specifies the database within
each driver. PR #607 introduced a Neo4j compatability, as the database
names are different when attempting to support FalkorDB.

This refactor improves compatability across database types and ensures
future reliance by isolating the configuraiton to the driver level.

* fix: make falkordb support optional

This ensures that the the optional dependency and subsequent import is compliant with the graphiti-core project dependencies.

* chore: fmt code

* chore: undo changes to uv.lock

* fix: undo potentially breaking changes to drive interface

* fix: ensure a default database of "None" is provided - falling back to internal default

* chore: ensure default value exists for session and delete_all_indexes

* chore: fix typos and grammar

* chore: update package versions and dependencies in uv.lock and bulk_utils.py

* docs: update database configuration instructions for Neo4j and FalkorDB

Clarified default database names and how to override them in driver constructors. Updated testing requirements to include specific commands for running integration and unit tests.

* fix: ensure params defaults to an empty dictionary in Neo4jDriver

Updated the execute_query method to initialize params as an empty dictionary if not provided, ensuring compatibility with the database configuration.

---------

Co-authored-by: Urmzd <urmzd@dal.ca>
2025-07-10 17:25:39 -04:00
Preston Rasmussen
71360d91fc
reformat (#655) 2025-07-01 12:26:15 -04:00
Daniel Chalef
689d669559
Gemini client improvements; Gemini reranker (#645)
* add support for Gemini 2.5 model thinking budget

* allow adding thinking config to support current and future gemini models

* merge

* improve client; add reranker

* refactor: change type hint for gemini_messages to Any for flexibility

* refactor: update GeminiRerankerClient to use direct relevance scoring and improve ranking logic. Add tests

* fix fixtures

---------

Co-authored-by: realugbun <github.disorder751@passmail.net>
2025-06-30 12:55:17 -07:00
Daniel Chalef
8213d10d44
migrate to pyright (#646)
* migrate to pyright

* Refactor type checking to use Pyright, update dependencies, and clean up code.

- Replaced MyPy with Pyright in configuration files and CI workflows.
- Updated `pyproject.toml` and `uv.lock` to reflect new dependencies and versions.
- Adjusted type hints and fixed minor code issues across various modules for better compatibility with Pyright.
- Added new packages `backoff` and `posthog` to the project dependencies.

* Update CI workflows to install all extra dependencies for type checking and unit tests

* Update dependencies in uv.lock to replace MyPy with Pyright and add nodeenv package. Adjust type hinting in config.py for compatibility with Pyright.
2025-06-30 12:04:21 -07:00
prestonrasmussen
7a8283dbac make falkor tests optional 2025-06-30 11:26:30 -04:00
prestonrasmussen
885d14c9f8 make unit tests optional 2025-06-30 11:18:12 -04:00
Gal Shubeli
6e6115c134
FalkorDB Integration: Bug Fixes and Unit Tests (#607)
* fixes-and-tests

* update-workflow

* lint-fixes

* mypy-fixes

* fix-falkor-tests

* Update poetry.lock after pyproject.toml changes

* update-yml

* fix-tests

* comp-tests

* typo

* fix-tests

---------

Co-authored-by: Guy Korland <gkorland@gmail.com>
2025-06-30 11:01:44 -04:00
Daniel Chalef
c29893d972
Excluded entity type filtering (#624)
* excluded entities filtering

* Fix variable name casing in test_entity_exclusion_int.py for consistency
2025-06-26 20:54:43 -07:00
Preston Rasmussen
9422b6f5fb
Node dedupe efficiency (#490)
* update resolve extracted edge

* updated edge resolution

* dedupe nodes update

* single pass node resolution

* updates

* mypy updates

* Update graphiti_core/prompts/dedupe_nodes.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* remove unused imports

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-15 13:56:33 -04:00
Preston Rasmussen
9baa9b7b8a
Mmr optimizations (#481)
* update mmr calculations

* update search

* fixes and updates

* mypy
2025-05-12 22:30:23 -04:00
prestonrasmussen
f7b2c3845b test fix 2025-05-08 00:27:05 -04:00
Preston Rasmussen
8bdceaea55
load embedding update (#458)
update
2025-05-08 00:07:53 -04:00
Preston Rasmussen
fd9969b5a1
Update dedupe prompt (#457)
* improve dedupe logic

* cut summary length

* update unit tests
2025-05-07 23:23:31 -04:00
Evan Schultz
5baaa6fa8c
Anthropic cleanup (#431)
* remove temporary debug logging

* add anthropic api to .env.example

* move anthropic int tests to llm_client dir to better match existing test structure

* update `TestLLMClient` to `MockLLMClient` to eliminate pytest warning
2025-05-03 09:15:03 -04:00
Evan Schultz
f2e95a5685
add embedder tests (#430) 2025-05-03 09:13:56 -04:00
Preston Rasmussen
1193b25fa3
add_episode() refactor (#421)
* temporal updates

* update resolve nodes

* dedupe edge updates

* edge dedupe

* extract attributes

* update dynamic pydantic model

* first pass of extract node attributes

* no errors

* bug fixes

* bug fixes

* prompt updates

* prompt updates

* updates

* updates

* remove unused imports

* update tests based on changes

* remove unused import
2025-04-30 12:08:52 -04:00
Preston Rasmussen
7ee4e38616
Eval updates (#404)
* update eval

* make format

* remove unused imports

* mypy
2025-04-27 14:27:47 -04:00
Preston Rasmussen
009467650f
Node episodes list (#381)
* added episode list virtual field

* in progress tests

* add tests

* update search return type

* linter

* copyright notice

* mark integration tests
2025-04-20 23:20:19 -04:00
Evan Schultz
113179f674
Anthropic client (#361)
* update Anthropic client to use tool calling and add tests

* fix linting errors before creating pull request by making literal types for anthropic models
2025-04-16 12:35:07 -07:00
Preston Rasmussen
45b15a06f2
add episode scope to search (#362)
* add episode scope to search

* bump version

* linter

* Update graphiti_core/search/search_helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-15 19:27:56 -04:00
Preston Rasmussen
11e19a35b7
add reranker_min_score (#355)
* add reranker_min_score

* update divide by 0 case

* center node always gets a score of .1

* linter
2025-04-15 12:33:37 -04:00
Preston Rasmussen
c8d5c45269
update to 4.1 models (#352) 2025-04-14 21:02:36 -04:00
Preston Rasmussen
5dce26722e
e2e graph builder eval (#343)
* add partial eval platform

* dedupe updates

* add e2e eval

* Update graphiti_core/prompts/eval.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* clear all outputs

* clear all outputs

* squash eval commits

* Update tests/evals/data/utils.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* add longmemeval disclaimer

* remove gitignore

* add copyright headers

* add cli

* Update tests/evals/data/longmemeval_data/README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update tests/evals/eval_e2e_graph_building.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* updates

---------

Co-authored-by: jackaldenryan <jackaldenryan@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-12 10:35:22 -04:00
Preston Rasmussen
6aa25a1901
update context string (#346)
* update context string

* Update graphiti_core/search/search_helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* remove unused imports

* bump version

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-10 06:57:58 -04:00
Daniel Chalef
0f6ac57dab
chore: update version to 0.9.3 and restructure dependencies (#338)
* Bump version from 0.9.0 to 0.9.1 in pyproject.toml and update google-genai dependency to >=0.1.0

* Bump version from 0.9.1 to 0.9.2 in pyproject.toml

* Update google-genai dependency version to >=0.8.0 in pyproject.toml

* loc file

* Update pyproject.toml to version 0.9.3, restructure dependencies, and modify author format. Remove outdated Google API key note from README.md.

* upgrade poetry and ruff
2025-04-08 20:47:38 -07:00
Daniel Chalef
9e78890f2e
Gemini support (#324)
* first cut

* Update dependencies and enhance README for optional LLM providers

- Bump aiohttp version from 3.11.14 to 3.11.16
- Update yarl version from 1.18.3 to 1.19.0
- Modify pyproject.toml to include optional extras for Anthropic, Groq, and Google Gemini
- Revise README.md to reflect new optional LLM provider installation instructions and clarify API key requirements

* Remove deprecated packages from poetry.lock and update content hash

- Removed cachetools, google-auth, google-genai, pyasn1, pyasn1-modules, rsa, and websockets from the lock file.
- Added new extras for anthropic, google-genai, and groq.
- Updated content hash to reflect changes.

* Refactor import paths for GeminiClient in README and __init__.py

- Updated import statement in README.md to reflect the new module structure for GeminiClient.
- Removed GeminiClient from the __all__ list in __init__.py as it is no longer directly imported.

* Refactor import paths for GeminiEmbedder in README and __init__.py

- Updated import statement in README.md to reflect the new module structure for GeminiEmbedder.
- Removed GeminiEmbedder and GeminiEmbedderConfig from the __all__ list in __init__.py as they are no longer directly imported.
2025-04-06 09:27:04 -07:00
Preston Rasmussen
1d2417ec26
Search optimizations (#280)
fix node distance search
2025-02-27 11:51:10 -05:00
Preston Rasmussen
9efa6762d7
entity typo (#274) 2025-02-24 12:44:17 -05:00
Preston Rasmussen
088029a80c
node label filters (#265)
* node label filters

* update

* add search filters

* updates

* bump versions

* update tests

* test update
2025-02-21 12:38:01 -05:00
Preston Rasmussen
00fe87679e
Bounded semaphore - limiting concurrency (#244)
* WIP

* add semaphore

* remove unused imports

* remove unused imports

* lower concurrency limit
2024-12-17 13:08:18 -05:00
Pavlo Paliychuk
a9091b06ff
fix: Clean input before passing it to the llm (#238)
* fix: Clean input before passing it to the llm

* chore: Add license

* fix: typo

* chore: Bump graphiti version
2024-12-10 21:27:05 -05:00
Daniel Chalef
445dccc021
refactor: use utc_now() for consistent UTC datetime handling (#234)
* ensure utc timezones

* fix: dep cycle

---------

Co-authored-by: paulpaliychuk <pavlo.paliychuk.ca@gmail.com>
2024-12-09 10:36:04 -08:00
Preston Rasmussen
732b2f328d
update lucene escaping (#233)
* update lucene escaping

* update unit test
2024-12-09 10:36:46 -05:00
Preston Rasmussen
6a152ab91a
fix node distance reranker (#231) 2024-12-06 12:08:54 -05:00
Preston Rasmussen
397291de4b
add delete nodes by group_id (#217)
* add delete nodes by group_id

* remove unused imports

* bump version
2024-11-18 13:55:18 -05:00
Preston Rasmussen
281fe072cb
add fulltext search limit (#215)
* add fulltext search limit

* format

* update

* update

* update tests

* remove unused imports

* format

* mypy
2024-11-14 12:18:18 -05:00
Preston Rasmussen
3199e893ed
add_fact endpoint (#207)
* add_fact endpoint

* bump version

* add edge invalidation

* update
2024-11-06 09:12:21 -05:00
Preston Rasmussen
ceb60a3d33
Cross encoder reranker in search query (#202)
* cross encoder reranker

* update reranker

* add openai reranker

* format

* mypy

* update

* updates

* MyPy typing

* bump version
2024-10-25 12:29:27 -04:00
Pavlo Paliychuk
544f9e3fba
chore: Set up cross encoder client (#201)
* chore: Set up cross encoder client

* fix: deps

* chore: move voyage to dev deps
2024-10-24 11:36:10 -04:00
Preston Rasmussen
b217d1e51f
Make default DB explicit (#195)
* add default database

* update

* init tests

* update test

* bump version

* removed unused imports
2024-10-21 12:33:32 -04:00
Preston Rasmussen
49aeaf75f2
Add mmr reranking (#180)
* mmr start

* add mmr function

* normalize

* add mmr options to search

* update communities

* build communities

* format

* clean up normalization

* normalize in mmr

* update
2024-10-08 13:55:10 -04:00
Preston Rasmussen
e15c872900
Fix edge invalidation (#174)
* update edge operations

* add new tests
2024-10-07 11:45:31 -04:00
Preston Rasmussen
ec2e51c5ec
test escape characters (#171)
* test escape characters

* format

* tests

* run tests

* copyright
2024-10-03 10:08:30 -04:00
Pavlo Paliychuk
a7148d6260
feat: Dedicated embedder interface (#159)
* feat: Add Embedder interface and implement openai embedder

* feat: Add voyage ai embedder
2024-09-27 12:47:04 -04:00
ARNO
5bd18fc7dd
feat: configurable embedding model (#156)
* feat: configurable embedding model

format

* chore: Update comment

* chore: Pass embedding model in search utils

---------

Co-authored-by: paulpaliychuk <pavlo.paliychuk.ca@gmail.com>
2024-09-26 13:31:22 -07:00
Preston Rasmussen
fd341a6f16
Add MSC benchmark and improve search performance (#157)
* test cases

* test

* benchmark

* eval updates

* improve search performance

* remove data

* formatting

* add None type to config

* update sanitization

* push version

* maketrans update

* mypy
2024-09-26 16:12:38 -04:00