Commit graph

272 commits

Author SHA1 Message Date
Daniel Chalef
d7828d48d8
Fix index out of range errors in LLM deduplication responses (#939)
* add tests for llm dedupe guardrails

* document llm dedupe guardrails
2025-09-26 14:57:48 -07:00
Daniel Chalef
9aee3174bd
Refactor batch deduplication logic to enhance node resolution and track duplicate pairs (#929) (#936)
* Refactor deduplication logic to enhance node resolution and track duplicate pairs (#929)

* Simplify deduplication process in bulk_utils by reusing canonical nodes.
* Update dedup_helpers to store duplicate pairs during resolution.
* Modify node_operations to append duplicate pairs when resolving nodes.
* Add tests to verify deduplication behavior and ensure correct state updates.

* reveret to concurrent dedup with fanout and then reconcilation

* add performance note for deduplication loop in bulk_utils

* enhance deduplication logic in bulk_utils to handle missing canonical nodes gracefully

* Update graphiti_core/utils/bulk_utils.py

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* refactor deduplication logic in bulk_utils to use directed union-find for canonical UUID resolution

* implement _build_directed_uuid_map for efficient UUID resolution in bulk_utils

* document directed union-find lookup in bulk_utils for clarity

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-09-26 08:40:18 -07:00
Daniel Chalef
7c469e8e2b
Improve node deduplication w/ deterministic matching, LLM fallbacks (#929)
* add repository guidelines and project structure documentation

* update neo4j image version and modify test command to disable specific databases

* implement deduplication helpers and integrate with node operations

* refactor string formatting to use single quotes in node operations

* enhance deduplication helpers with UUID indexing and update resolution logic

* implement exact fact matching (#931)
2025-09-25 07:13:19 -07:00
Preston Rasmussen
d6d4bbdeb7
don't save duplicate edges (#927)
* don't save duplicate edges

* remove build duplicate edges
2025-09-24 17:24:57 -04:00
Daniel Chalef
7cf5ee6288
Skip entity attribute extraction when no fields defined (#924) 2025-09-24 13:23:37 -04:00
Preston Rasmussen
36056ad141
Graph quality updates (#922)
duplicate_of updates
2025-09-23 17:53:39 -04:00
Gal Shubeli
d725fcdf8e
fix-fulltext-syntax-error (#914)
* fix-fulltext-syntax-error

* update-abs-method
2025-09-23 10:52:44 -04:00
Preston Rasmussen
da71d118db
Embedding fix (#917)
* embedding fix

* pre3

* fixedmake format
2025-09-20 09:00:04 -04:00
Preston Rasmussen
3efe085a92
OpenSearch updates (#906)
* updates

* add uuid filter functionality

* update

* updates

* bump-version

* update

* fix typo

* use async function

* update unit tests

* update delete

* update deletion

* async update

* update

* update

* update

* update
2025-09-14 01:43:37 -04:00
Preston Rasmussen
0884cc00e5
OpenSearch Integration for Neo4j (#896)
* move aoss to driver

* add indexes

* don't save vectors to neo4j with aoss

* load embeddings from aoss

* add group_id routing

* add search filters and similarity search

* neptune regression update

* update neptune for regression purposes

* update index creation with aliasing

* regression tested

* update version

* edits

* claude suggestions

* cleanup

* updates

* add embedding dim env var

* use cosine sim

* updates

* updates

* remove unused imports

* update
2025-09-09 10:51:46 -04:00
Preston Rasmussen
ce1ae30569
Add return to add_triplet (#898)
* update

* add triplet results

* Update graphiti_core/graphiti.py

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-09-08 15:39:05 -04:00
Preston Rasmussen
7e6d93fa32
add episode bulk search results (#897)
* add episode bulk search results

* update

* docstring

* update
2025-09-08 14:34:32 -04:00
Preston Rasmussen
1f5a1b890c
cleanup (#894)
* cleanup

* update

* remove unused imports
2025-09-05 11:30:46 -04:00
Preston Rasmussen
eeb0d877de
update (#891) 2025-09-03 18:42:58 -04:00
Preston Rasmussen
81d110f944
bump version (#889)
* bump version

* remove unused imports
2025-09-03 14:08:35 -04:00
prestonrasmussen
29ba336189 remove parallel runtime and build dynamic indexes sequentially 2025-09-03 13:53:12 -04:00
Preston Rasmussen
1460172568
don't return index labels (#887)
* don't return index labels

* update tests
2025-09-02 12:02:33 -04:00
Preston Rasmussen
da6f3336bb
update-tests (#872)
* update-tests

* unit test update

* update tests

* update tests

* update kuzu query

* update

* update query

* update args

* fix bulk episode add

* make handling better
2025-08-31 13:19:29 -04:00
Siddhartha Sahu
8802b7db13
Add support for Kuzu as the graph driver (#799)
* Fix FalkoDB tests

* Add support for graph memory using Kuzu

* Fix lints

* Fix queries

* Add tests

* Add comments

* Add more test coverage

* Add mocked tests

* Format

* Add mocked tests II

* Refactor community queries

* Add more mocked tests

* Refactor tests to always cleanup

* Add more mocked tests

* Update kuzu

* Refactor how filters are built

* Add more mocked tests

* Refactor and cleanup

* Fix tests

* Fix lints

* Refactor tests

* Disable neptune

* Fix

* Update kuzu version

* Update kuzu to latest release

* Fix filter

* Fix query

* Fix Neptune query

* Fix bulk queries

* Fix lints

* Fix deletes

* Comments and format

* Add Kuzu to the README

* Fix bulk queries

* Test all fields of nodes and edges

* Fix lints

* Update search_utils.py

---------

Co-authored-by: Preston Rasmussen <109292228+prasmussen15@users.noreply.github.com>
2025-08-27 11:45:21 -04:00
Preston Rasmussen
309159bccb
update migration (#870)
* update migration

* bump version

* close driver
2025-08-27 11:13:10 -04:00
bechbd
41c3da2440
Fixed issue where creating indices was not called for Neptune and added missing quickstart example (#850)
* Rebased Neptune changes based on significant rework done

* Updated the README documentation

* Fixed linting and formatting

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/driver/neptune_driver.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Addressed feedback from code review

* Updated the README documentation for clarity

* Updated the README and neptune_driver based on PR feedback

* Update node_db_queries.py

* bug: Fixed issue with missing call to create indicies for Neptune and added quickstart example

* chore: added pyright to ignore the attribute not in GrapHDriver

* Fixed quickstart with feedback from automated PR

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Preston Rasmussen <109292228+prasmussen15@users.noreply.github.com>
2025-08-26 11:51:20 -04:00
Preston Rasmussen
fa9c1696b8
dont create extra search embeddings (#861)
* dont create extra search embeddings

* updates

* add missing conditionals

* fix

* float 0

* null check

* more nullchecks

* bump version
2025-08-26 11:16:46 -04:00
Preston Rasmussen
0ac7ded4d1
use hnsw indexes (#859)
* use hnsw indexes

* add migration

* updates

* add group_id validation

* updates

* add type annotation

* updates

* update

* swap to prerelease
2025-08-25 12:31:35 -04:00
Preston Rasmussen
1edcbaa9e9
Gpt 5 default (#849)
* gpt-5-mini and gpt-5-nano default

* bump version

* remove unused imports

* linter

* update

* disable neptune errors while we get a fixture in place

* update pyright

* revert non-structured completions

* fix typo
2025-08-21 12:10:57 -04:00
bechbd
ef56dc779a
Amazon Neptune Support (#793)
* Rebased Neptune changes based on significant rework done

* Updated the README documentation

* Fixed linting and formatting

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/driver/neptune_driver.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Addressed feedback from code review

* Updated the README documentation for clarity

* Updated the README and neptune_driver based on PR feedback

* Update node_db_queries.py

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Preston Rasmussen <109292228+prasmussen15@users.noreply.github.com>
2025-08-20 10:56:03 -04:00
Preston Rasmussen
1c27a3563b
update prompts and support thinking models (#846)
* update prompts and support thinking models

* update

* type ignore
2025-08-19 12:31:50 -04:00
Preston Rasmussen
c28bde6b07
fix typo and model selector (#843)
* fix typo and model selector

* bump version
2025-08-18 11:15:45 -04:00
Gal Shubeli
1abb4b0fa3
Fix Community Operations with FalkorDB (#824)
* Update node_db_queries.py

* Update node_db_queries.py

* fix-community-operations

---------

Co-authored-by: Naseem Ali <34807727+Naseem77@users.noreply.github.com>
2025-08-18 10:38:24 -04:00
Preston Rasmussen
1278f877d8
add bulk delete (#837)
* add bulk delete

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 12:15:07 -04:00
Cklee
506bc4fc98
Fix: Search methods configuration ignored (#788) (#829)
fix: respect search_methods configuration in edge_search and node_search

The search functions were ignoring the configured search_methods and
always executing all three methods (BM25, cosine similarity, and BFS).
This was causing unnecessary performance overhead - running 3x the
required computations when users only wanted specific search methods.

Now the functions build search tasks dynamically based on what's
actually configured in search_methods, only executing the requested
search algorithms.

This should significantly improve performance when users need only
specific search methods. Would appreciate your review on this approach\!

Fixes #788
2025-08-13 11:56:20 -04:00
Preston Rasmussen
bfe51a0fcd
Null search datetimes (#818)
* support null operations

* update

* only provide variables for non-null params

* test update
2025-08-12 12:24:37 -04:00
Preston Rasmussen
baa6825708
ensure ascii default to false (#817) 2025-08-08 11:20:02 -04:00
HUGO SON
ce9ef3ca79
Add support for non-ASCII characters in LLM prompts (#805)
* Add support for non-ASCII characters in LLM prompts

- Add ensure_ascii parameter to Graphiti class (default: True)
- Create to_prompt_json helper function for consistent JSON serialization
- Update all prompt files to use new helper function
- Preserve Korean/Japanese/Chinese characters when ensure_ascii=False
- Maintain backward compatibility with existing behavior

Fixes issue where non-ASCII characters were escaped as unicode sequences
in prompts, making them unreadable in LLM logs and potentially affecting
model understanding.

* Remove unused json imports after replacing with to_prompt_json helper

- Fix ruff lint errors (F401) for unused json imports
- All prompt files now use to_prompt_json helper instead of json.dumps
- Maintains clean code style and passes lint checks

* Fix ensure_ascii propagation to all LLM calls

- Add ensure_ascii parameter to maintenance operation functions that were missing it
- Update function signatures in node_operations, community_operations, temporal_operations, and edge_operations
- Ensure all llm_client.generate_response calls receive proper ensure_ascii context
- Fix hardcoded ensure_ascii: True values that prevented non-ASCII character preservation
- Maintain backward compatibility with default ensure_ascii=True
- Complete the fix for issue #804 ensuring Korean/Japanese/Chinese characters are properly handled in LLM prompts
2025-08-08 11:07:32 -04:00
Preston Rasmussen
0c01417e1f
add batch delete capabilities (#813)
* add batch delete capabilities

* use session for delete queries
2025-08-07 15:51:30 -04:00
Preston Rasmussen
6f3f369be4
update add triple to always have embeddings (#803)
update
2025-08-04 16:33:07 -04:00
Preston Rasmussen
0eb98647a9
add community group id index (#802)
* add community group id index

* updates nodes and indices
2025-08-04 12:24:44 -04:00
Preston Rasmussen
ab8106cb4f
move summary out of attribute extraction (#792)
* move summary out of attribute extraction

* linter

* linter

* fix db query
2025-07-31 12:15:21 -04:00
prestonrasmussen
6f3a4f19eb dedupe prompt update 2025-07-29 18:43:42 -04:00
Preston Rasmussen
19bddb5528
validate pydantic objects (#783)
* validate pydantic objects

* unused imports

* linter
2025-07-29 17:54:09 -04:00
Daniel Chalef
dcc9da3f68
chore/prepare kuzu integration (#762)
* Prepare code

* Fix tests

* As -> AS, remove trailing spaces

* Enable more tests for FalkorDB

* Fix more cypher queries

* Return all created nodes and edges

* Add Neo4j service to unit tests workflow

- Introduced Neo4j as a service in the GitHub Actions workflow for unit tests.
- Configured Neo4j with appropriate ports, authentication, and health checks.
- Updated test steps to include waiting for Neo4j and running integration tests against it.
- Set environment variables for Neo4j connection in both non-integration and integration test steps.

* Update Neo4j authentication in unit tests workflow

- Changed Neo4j authentication password from 'test' to 'testpass' in the GitHub Actions workflow.
- Updated health check command to reflect the new password.
- Ensured consistency across all test steps that utilize Neo4j credentials.

* fix health check

* Fix Neo4j integration tests in CI workflow

Remove reference to non-existent test_neo4j_driver.py file from test command.
Integration tests now run via parametrized tests using the drivers list.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add OPENAI_API_KEY to Neo4j integration tests

Neo4j integration tests require OpenAI API access for LLM functionality.
Add the secret environment variable to enable these tests to run properly.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Neo4j Cypher syntax error in BFS search queries

Replace parameter substitution in relationship pattern ranges (*1..$depth)
with direct string interpolation (*1..{bfs_max_depth}). Neo4j doesn't allow
parameter maps in MATCH pattern ranges - they must be literal values.

Fixed in both node_bfs_search and edge_bfs_search functions.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix variable name mismatch in edge_bfs_search query

Change relationship variable from 'r' to 'e' to match ENTITY_EDGE_RETURN
constant expectations. The ENTITY_EDGE_RETURN constant references variable
'e' for relationships, but the query was using 'r', causing "Variable e
not defined" errors.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Isolate database tests in CI workflow

- FalkorDB tests: Add DISABLE_NEO4J=1 and remove Neo4j env vars
- Neo4j tests: Keep current setup without DISABLE_NEO4J flag

This ensures proper test isolation where each test suite only runs
against its intended database backend.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Siddhartha Sahu <sid@kuzudb.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-29 09:07:34 -04:00
Daniel Chalef
721e92f8fb
feat/falkordb dynamic graph names (#761)
* graphiti-graph-name

* fix-lint

* fix-unittest

* clone-update

* groupid-none

* groupid-def-fulltext

* lint

* Remove redundant function definition for fulltext_query in search_utils.py

* Refactor get_default_group_id function and remove redundant code in falkordb_driver and search_utils. Added import statement in driver.py.

* Refactor test cases in test_falkordb_driver.py for improved readability by consolidating multi-line assertions into single lines. No functional changes made.

* Refactor fulltext_query function in search_utils.py to use double quotes for group_id in the filter list, enhancing consistency in query syntax.

* Remove duplicate assignment of fuzzy_query in episode_fulltext_search function in search_utils.py to eliminate redundancy.

* Remove duplicate assignment of fuzzy_query in community_fulltext_search function in search_utils.py to streamline code.

---------

Co-authored-by: Gal Shubeli <galshubeli93@gmail.com>
2025-07-24 07:17:59 -07:00
Preston Rasmussen
17747ff58d
Return reranker scores (#758)
* add search reranker scores to search output

* bump version

* updates
2025-07-23 16:05:48 -04:00
Daniel Chalef
059a64e5e9
Group ID filtering in BFS and full-text queries (#754)
Enhance search functions to include group ID filtering in BFS and full-text queries
2025-07-22 17:02:46 -07:00
prestonrasmussen
5bbc3cf814 optimize fulltext query update 2025-07-22 14:12:21 -04:00
prestonrasmussen
c0cae61d52 fulltext query update 2025-07-22 14:07:41 -04:00
Preston Rasmussen
38dd3e8dc3
Edge search updates (#753)
* update edge fulltext search

* update

* update
2025-07-22 10:05:58 -04:00
Daniel Chalef
7daa46067e
fix: missing group filter on node ft search (#752)
* Enhance full-text search to filter by group IDs in node queries

* Bump version to 0.17.8 in pyproject.toml
2025-07-21 22:57:27 -07:00
prestonrasmussen
f4dc7e2fba update max query length 2025-07-21 19:39:31 -04:00
Gal Shubeli
35e0692328
[Bug Fix] Fix the Group ID usage with FalkorDB (#733)
* groupid-none

* groupid-def-fulltext

* lint

* Update graphiti_core/helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-17 12:35:08 -04:00
Preston Rasmussen
0ac2541b35
make egg_operations more robust (#737)
update
2025-07-16 17:12:20 -04:00