Commit graph

87 commits

Author SHA1 Message Date
Siddhartha Sahu
8802b7db13
Add support for Kuzu as the graph driver (#799)
* Fix FalkoDB tests

* Add support for graph memory using Kuzu

* Fix lints

* Fix queries

* Add tests

* Add comments

* Add more test coverage

* Add mocked tests

* Format

* Add mocked tests II

* Refactor community queries

* Add more mocked tests

* Refactor tests to always cleanup

* Add more mocked tests

* Update kuzu

* Refactor how filters are built

* Add more mocked tests

* Refactor and cleanup

* Fix tests

* Fix lints

* Refactor tests

* Disable neptune

* Fix

* Update kuzu version

* Update kuzu to latest release

* Fix filter

* Fix query

* Fix Neptune query

* Fix bulk queries

* Fix lints

* Fix deletes

* Comments and format

* Add Kuzu to the README

* Fix bulk queries

* Test all fields of nodes and edges

* Fix lints

* Update search_utils.py

---------

Co-authored-by: Preston Rasmussen <109292228+prasmussen15@users.noreply.github.com>
2025-08-27 11:45:21 -04:00
Preston Rasmussen
0ac7ded4d1
use hnsw indexes (#859)
* use hnsw indexes

* add migration

* updates

* add group_id validation

* updates

* add type annotation

* updates

* update

* swap to prerelease
2025-08-25 12:31:35 -04:00
Preston Rasmussen
1278f877d8
add bulk delete (#837)
* add bulk delete

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 12:15:07 -04:00
Preston Rasmussen
bfe51a0fcd
Null search datetimes (#818)
* support null operations

* update

* only provide variables for non-null params

* test update
2025-08-12 12:24:37 -04:00
Preston Rasmussen
baa6825708
ensure ascii default to false (#817) 2025-08-08 11:20:02 -04:00
HUGO SON
ce9ef3ca79
Add support for non-ASCII characters in LLM prompts (#805)
* Add support for non-ASCII characters in LLM prompts

- Add ensure_ascii parameter to Graphiti class (default: True)
- Create to_prompt_json helper function for consistent JSON serialization
- Update all prompt files to use new helper function
- Preserve Korean/Japanese/Chinese characters when ensure_ascii=False
- Maintain backward compatibility with existing behavior

Fixes issue where non-ASCII characters were escaped as unicode sequences
in prompts, making them unreadable in LLM logs and potentially affecting
model understanding.

* Remove unused json imports after replacing with to_prompt_json helper

- Fix ruff lint errors (F401) for unused json imports
- All prompt files now use to_prompt_json helper instead of json.dumps
- Maintains clean code style and passes lint checks

* Fix ensure_ascii propagation to all LLM calls

- Add ensure_ascii parameter to maintenance operation functions that were missing it
- Update function signatures in node_operations, community_operations, temporal_operations, and edge_operations
- Ensure all llm_client.generate_response calls receive proper ensure_ascii context
- Fix hardcoded ensure_ascii: True values that prevented non-ASCII character preservation
- Maintain backward compatibility with default ensure_ascii=True
- Complete the fix for issue #804 ensuring Korean/Japanese/Chinese characters are properly handled in LLM prompts
2025-08-08 11:07:32 -04:00
Preston Rasmussen
6f3f369be4
update add triple to always have embeddings (#803)
update
2025-08-04 16:33:07 -04:00
Preston Rasmussen
ab8106cb4f
move summary out of attribute extraction (#792)
* move summary out of attribute extraction

* linter

* linter

* fix db query
2025-07-31 12:15:21 -04:00
Daniel Chalef
dcc9da3f68
chore/prepare kuzu integration (#762)
* Prepare code

* Fix tests

* As -> AS, remove trailing spaces

* Enable more tests for FalkorDB

* Fix more cypher queries

* Return all created nodes and edges

* Add Neo4j service to unit tests workflow

- Introduced Neo4j as a service in the GitHub Actions workflow for unit tests.
- Configured Neo4j with appropriate ports, authentication, and health checks.
- Updated test steps to include waiting for Neo4j and running integration tests against it.
- Set environment variables for Neo4j connection in both non-integration and integration test steps.

* Update Neo4j authentication in unit tests workflow

- Changed Neo4j authentication password from 'test' to 'testpass' in the GitHub Actions workflow.
- Updated health check command to reflect the new password.
- Ensured consistency across all test steps that utilize Neo4j credentials.

* fix health check

* Fix Neo4j integration tests in CI workflow

Remove reference to non-existent test_neo4j_driver.py file from test command.
Integration tests now run via parametrized tests using the drivers list.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add OPENAI_API_KEY to Neo4j integration tests

Neo4j integration tests require OpenAI API access for LLM functionality.
Add the secret environment variable to enable these tests to run properly.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Neo4j Cypher syntax error in BFS search queries

Replace parameter substitution in relationship pattern ranges (*1..$depth)
with direct string interpolation (*1..{bfs_max_depth}). Neo4j doesn't allow
parameter maps in MATCH pattern ranges - they must be literal values.

Fixed in both node_bfs_search and edge_bfs_search functions.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix variable name mismatch in edge_bfs_search query

Change relationship variable from 'r' to 'e' to match ENTITY_EDGE_RETURN
constant expectations. The ENTITY_EDGE_RETURN constant references variable
'e' for relationships, but the query was using 'r', causing "Variable e
not defined" errors.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Isolate database tests in CI workflow

- FalkorDB tests: Add DISABLE_NEO4J=1 and remove Neo4j env vars
- Neo4j tests: Keep current setup without DISABLE_NEO4J flag

This ensures proper test isolation where each test suite only runs
against its intended database backend.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Siddhartha Sahu <sid@kuzudb.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-29 09:07:34 -04:00
Daniel Chalef
721e92f8fb
feat/falkordb dynamic graph names (#761)
* graphiti-graph-name

* fix-lint

* fix-unittest

* clone-update

* groupid-none

* groupid-def-fulltext

* lint

* Remove redundant function definition for fulltext_query in search_utils.py

* Refactor get_default_group_id function and remove redundant code in falkordb_driver and search_utils. Added import statement in driver.py.

* Refactor test cases in test_falkordb_driver.py for improved readability by consolidating multi-line assertions into single lines. No functional changes made.

* Refactor fulltext_query function in search_utils.py to use double quotes for group_id in the filter list, enhancing consistency in query syntax.

* Remove duplicate assignment of fuzzy_query in episode_fulltext_search function in search_utils.py to eliminate redundancy.

* Remove duplicate assignment of fuzzy_query in community_fulltext_search function in search_utils.py to streamline code.

---------

Co-authored-by: Gal Shubeli <galshubeli93@gmail.com>
2025-07-24 07:17:59 -07:00
Preston Rasmussen
17747ff58d
Return reranker scores (#758)
* add search reranker scores to search output

* bump version

* updates
2025-07-23 16:05:48 -04:00
Gal Shubeli
35e0692328
[Bug Fix] Fix the Group ID usage with FalkorDB (#733)
* groupid-none

* groupid-def-fulltext

* lint

* Update graphiti_core/helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-17 12:35:08 -04:00
Preston Rasmussen
5d45d71259
Bulk updates (#732)
* updates

* update

* update

* typo

* linter
2025-07-16 02:26:33 -04:00
Daniel Chalef
aa6e38856a
[REFACTOR][FIX] Move away from DEFAULT_DATABASE environment variable in favour of driver-config support (dc) (#699)
* fix: remove global DEFAULT_DATABASE usage in favor of driver-specific
config

Fixes bugs introduced in PR #607. This removes reliance on the global
DEFAULT_DATABASE environment variable. It specifies the database within
each driver. PR #607 introduced a Neo4j compatability, as the database
names are different when attempting to support FalkorDB.

This refactor improves compatability across database types and ensures
future reliance by isolating the configuraiton to the driver level.

* fix: make falkordb support optional

This ensures that the the optional dependency and subsequent import is compliant with the graphiti-core project dependencies.

* chore: fmt code

* chore: undo changes to uv.lock

* fix: undo potentially breaking changes to drive interface

* fix: ensure a default database of "None" is provided - falling back to internal default

* chore: ensure default value exists for session and delete_all_indexes

* chore: fix typos and grammar

* chore: update package versions and dependencies in uv.lock and bulk_utils.py

* docs: update database configuration instructions for Neo4j and FalkorDB

Clarified default database names and how to override them in driver constructors. Updated testing requirements to include specific commands for running integration and unit tests.

* fix: ensure params defaults to an empty dictionary in Neo4jDriver

Updated the execute_query method to initialize params as an empty dictionary if not provided, ensuring compatibility with the database configuration.

---------

Co-authored-by: Urmzd <urmzd@dal.ca>
2025-07-10 17:25:39 -04:00
Preston Rasmussen
0675ac2b7d
Bulk ingestion (#698)
* partial

* update

* update

* update

* update

* updates

* updates

* update

* update
2025-07-10 12:14:49 -04:00
Preston Rasmussen
71360d91fc
reformat (#655) 2025-07-01 12:26:15 -04:00
Gal Shubeli
6e6115c134
FalkorDB Integration: Bug Fixes and Unit Tests (#607)
* fixes-and-tests

* update-workflow

* lint-fixes

* mypy-fixes

* fix-falkor-tests

* Update poetry.lock after pyproject.toml changes

* update-yml

* fix-tests

* comp-tests

* typo

* fix-tests

---------

Co-authored-by: Guy Korland <gkorland@gmail.com>
2025-06-30 11:01:44 -04:00
Daniel Chalef
cb4e187aed
feat: add telemetry with PostHog and update Docker configurations (#633)
* Implement telemetry feature for anonymous usage statistics collection in Graphiti; update Dockerfile CMD format for better signal handling; adjust Neo4j URI and healthcheck in docker-compose.yml; add new dependencies in pyproject.toml and poetry.lock.

* remove duplicated properties

* Update Dockerfile CMD to use JSON array format for improved signal handling

* remove tommlib dep only in 3.11

* Delete server/graph_service/logging_config.py
2025-06-27 12:23:30 -07:00
Daniel Chalef
c29893d972
Excluded entity type filtering (#624)
* excluded entities filtering

* Fix variable name casing in test_entity_exclusion_int.py for consistency
2025-06-26 20:54:43 -07:00
Daniel Chalef
a6bb9b3eca
Add group ID validation and error handling (#618)
- Introduced `GroupIdValidationError` to handle invalid group ID formats.
- Added `validate_group_id` function to check that group IDs contain only alphanumeric characters, dashes, or underscores.
- Integrated `validate_group_id` checks in the `Graphiti` class to ensure group IDs are validated during processing.
2025-06-24 09:33:54 -07:00
Daniel Chalef
fe870b953f
Add max_coroutines parameter to Graphiti and update semaphore_gather function (#619)
- Introduced max_coroutines parameter in the Graphiti class to control the maximum number of concurrent operations.
- Updated the semaphore_gather function to accept max_coroutines as an optional argument, defaulting to SEMAPHORE_LIMIT if not provided.
- Adjusted multiple calls to semaphore_gather throughout the Graphiti class to utilize the new max_coroutines parameter for better concurrency management.
2025-06-24 09:32:16 -07:00
Preston Rasmussen
2b0bc21b21
be more explicit about edge type signatures (#600)
* be more explicit about edge type signatures

* bump version

* update
2025-06-18 16:01:00 -04:00
Preston Rasmussen
e8bf81fc6b
add IS_DUPLICATE_OF edges (#599)
* add IS_DUPLICATE_OF edges

* cypher query update

* robust handling
2025-06-17 11:56:55 -04:00
Preston Rasmussen
19fde653a6
update driver (#583)
* update driver

* mypy updates

* mypy updates

* mypy updates

* Update graphiti_core/graph_queries.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy updates

* mypy

* mypy updates

* mypy updates

* mypy updates

* mypy updates

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-13 14:12:09 -04:00
Preston Rasmussen
14146dc46f
Add support for falkordb (#575)
* [wip] add support for falkordb

* updates

* fix-async

* progress

* fix-issues

* rm-date-handler

* red-code

* rm-uns-try

* fix-exm

* rm-un-lines

* fix-comments

* fix-se-utils

* fix-falkor-readme

* fix-falkor-cosine-score

* update-falkor-ver

* fix-vec-sim

* min-updates

* make format

* update graph driver abstraction

* poetry lock

* updates

* linter

* Update graphiti_core/search/search_utils.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: Dudi Zimberknopf <zimber.dudi@gmail.com>
Co-authored-by: Gal Shubeli <galshubeli93@gmail.com>
Co-authored-by: Gal Shubeli <124919062+galshubeli@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-13 12:06:57 -04:00
Preston Rasmussen
db7595fe63
Edge types (#501)
* update entity edge attributes

* Adding prompts

* extract fact attributes

* edge types

* edge types no regressions

* mypy

* mypy update

* Update graphiti_core/prompts/dedupe_edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update graphiti_core/prompts/dedupe_edges.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-19 13:30:56 -04:00
Preston Rasmussen
9422b6f5fb
Node dedupe efficiency (#490)
* update resolve extracted edge

* updated edge resolution

* dedupe nodes update

* single pass node resolution

* updates

* mypy updates

* Update graphiti_core/prompts/dedupe_nodes.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* remove unused imports

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-15 13:56:33 -04:00
Preston Rasmussen
89c4ee8cad
make bulk save more robust (#461)
* make bulk save more robust

* updates
2025-05-08 15:34:13 -04:00
prestonrasmussen
8ce9b1e157 fix bugs 2025-05-07 22:46:35 -04:00
Preston Rasmussen
2ffc58b3da
small model fix (#432)
* updated dedupe nodes operations

* updates

* Update examples/podcast/podcast_transcript.txt

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-02 10:08:25 -04:00
Preston Rasmussen
1193b25fa3
add_episode() refactor (#421)
* temporal updates

* update resolve nodes

* dedupe edge updates

* edge dedupe

* extract attributes

* update dynamic pydantic model

* first pass of extract node attributes

* no errors

* bug fixes

* bug fixes

* prompt updates

* prompt updates

* updates

* updates

* remove unused imports

* update tests based on changes

* remove unused import
2025-04-30 12:08:52 -04:00
Preston Rasmussen
a26b25dc06
Add episode refactor (#399)
* partial refactor

* get relevant nodes refactor

* load edges updates

* refactor triplets

* not there yet

* node search update

* working refactor

* updates

* mypy

* mypy
2025-04-26 00:24:23 -04:00
Pavlo Paliychuk
daaa0beab9
Fix edge deletion check to handle empty episodes list (#370) 2025-04-17 16:36:40 -04:00
Preston Rasmussen
45b15a06f2
add episode scope to search (#362)
* add episode scope to search

* bump version

* linter

* Update graphiti_core/search/search_helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-15 19:27:56 -04:00
Preston Rasmussen
502b6da1c7
Add search_ and deprecate _search (#342)
* add search_ and deprecate _search. Add formatting helper

* add search helpers file

* move SearchResults

* Update graphiti_core/search/search_helpers.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* remove unused imports

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-09 15:59:21 -04:00
Daniel Chalef
0f6ac57dab
chore: update version to 0.9.3 and restructure dependencies (#338)
* Bump version from 0.9.0 to 0.9.1 in pyproject.toml and update google-genai dependency to >=0.1.0

* Bump version from 0.9.1 to 0.9.2 in pyproject.toml

* Update google-genai dependency version to >=0.8.0 in pyproject.toml

* loc file

* Update pyproject.toml to version 0.9.3, restructure dependencies, and modify author format. Remove outdated Google API key note from README.md.

* upgrade poetry and ruff
2025-04-08 20:47:38 -07:00
Daniel Chalef
9e78890f2e
Gemini support (#324)
* first cut

* Update dependencies and enhance README for optional LLM providers

- Bump aiohttp version from 3.11.14 to 3.11.16
- Update yarl version from 1.18.3 to 1.19.0
- Modify pyproject.toml to include optional extras for Anthropic, Groq, and Google Gemini
- Revise README.md to reflect new optional LLM provider installation instructions and clarify API key requirements

* Remove deprecated packages from poetry.lock and update content hash

- Removed cachetools, google-auth, google-genai, pyasn1, pyasn1-modules, rsa, and websockets from the lock file.
- Added new extras for anthropic, google-genai, and groq.
- Updated content hash to reflect changes.

* Refactor import paths for GeminiClient in README and __init__.py

- Updated import statement in README.md to reflect the new module structure for GeminiClient.
- Removed GeminiClient from the __all__ list in __init__.py as it is no longer directly imported.

* Refactor import paths for GeminiEmbedder in README and __init__.py

- Updated import statement in README.md to reflect the new module structure for GeminiEmbedder.
- Removed GeminiEmbedder and GeminiEmbedderConfig from the __all__ list in __init__.py as they are no longer directly imported.
2025-04-06 09:27:04 -07:00
Preston Rasmussen
f4dd39128c
add entity type validation handling (#320)
* add entity type validation handling

* bump version
2025-04-04 13:53:04 -04:00
Preston Rasmussen
fb37565112
Retrieve episodes override (#319)
* add episode override

* add test

* make format
2025-04-02 17:19:24 -04:00
Preston Rasmussen
dff0b3c42e
get_nodes_and_edges_by_episode (#292)
update name
2025-03-13 15:50:03 -04:00
Preston Rasmussen
daf14a6509
get edges by node uuid (#291)
* get edges by node uuid

* lint
2025-03-13 15:44:02 -04:00
Preston Rasmussen
e83bcbb435
swap type_description to docstring (#287)
* swap type_description to docstring

* remove unused imports

* bump version

* removed unused imports
2025-03-05 15:27:03 -05:00
Preston Rasmussen
f73867e0fa
Entity classification updates (#285)
* updates

* tested

* remove unused imports

* llm outputs will be dicts rather than pydantic models

* removed unused imports
2025-03-05 12:08:11 -05:00
Preston Rasmussen
088029a80c
node label filters (#265)
* node label filters

* update

* add search filters

* updates

* bump versions

* update tests

* test update
2025-02-21 12:38:01 -05:00
Preston Rasmussen
29a071b2b8
Custom ontology (#262)
* ontology

* extract and save node labels

* extract entity type properties

* neo4j upgrade needed

* add entity types

* update typing

* update types

* updates

* Update graphiti_core/utils/maintenance/node_operations.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix warning

* mypy updates

* update properties

* mypy ignore

* mypy types

* bump version

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-13 12:17:52 -05:00
Preston Rasmussen
0e45d15462
Add triple update (#263)
* update add-triplet

* test fixes
2025-02-12 12:04:43 -05:00
Preston Rasmussen
a99aad59de
Remove Episode by uuid (#261)
* add remove_episode

* delete episodes

* update

* bump version
2025-02-05 15:17:08 -05:00
Preston Rasmussen
6ef2f5e097
Date filters (#240)
* add search filters

* add search filters

* mypy

* mypy

* update filtering

* date-filters

* update

* update filter queries

* update dictionary
2025-01-28 11:52:53 -05:00
Preston Rasmussen
d3b2cecbe5
WIP add episode bulk (#256) 2025-01-27 09:52:15 -05:00
Preston Rasmussen
00fe87679e
Bounded semaphore - limiting concurrency (#244)
* WIP

* add semaphore

* remove unused imports

* remove unused imports

* lower concurrency limit
2024-12-17 13:08:18 -05:00