Commit graph

27 commits

Author SHA1 Message Date
Daniel Chalef
196eb2f077
Remove JSON indentation from prompts to reduce token usage (#985)
Changes to `to_prompt_json()` helper to default to minified JSON (no indentation) instead of 2-space indentation. This reduces token consumption in LLM prompts while maintaining all necessary information.

- Changed default `indent` parameter from `2` to `None` in `prompt_helpers.py`
- Updated all prompt modules to remove explicit `indent=2` arguments
- Minor code formatting fixes in LLM clients

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-06 16:08:43 -07:00
Daniel Chalef
b28bd92c16
Remove ensure_ascii configuration parameter (#969)
* Remove ensure_ascii configuration parameter

- Changed to_prompt_json default from ensure_ascii=True to False
- Removed ensure_ascii parameter from Graphiti.__init__ and GraphitiClients
- Removed ensure_ascii from all function signatures and context dictionaries
- Removed ensure_ascii from all test files
- All JSON serialization now preserves Unicode characters by default

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-02 15:10:57 -07:00
Daniel Chalef
5ca8b9565c
fix: Improve deduplication ID validation and logging (#965)
* fix: Improve deduplication ID validation and logging

- Add comprehensive logging to verify IDs sent to LLM (sent vs received)
- Enhance prompt with explicit ID bounds (0 through N-1)
- Add validation warnings for missing and extra IDs from LLM responses
- Improve error message clarity for invalid dedupe IDs
- Log actual IDs sent to LLM to confirm no index leakage

This helps diagnose cases where the LLM returns IDs outside the valid
range (e.g., ID 19 when only 0-18 were sent).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Remove redundant logging parameter

Address reviewer comment about redundant third parameter in debug log statement.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address reviewer comments on list slicing and prompt clarity

- Fix list slicing bug: change <= to < to avoid gap when exactly 20 elements
  (previously would skip element 10 when showing 21 elements)
- Consolidate redundant prompt phrasing while maintaining clarity
  (reduced from 3 sentences to 2, keeping essential constraints)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Remove redundant prompt text to reduce token usage

Consolidate 'using these exact IDs (0 through N-1)' with following sentence
to eliminate repetition. Changes:
- 'using these exact IDs (0 through {N-1}). Do not skip IDs or use IDs outside this range'
- 'with IDs 0 through {N-1}. Do not skip or add IDs'

Saves ~15 tokens per deduplication call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-02 12:22:07 -07:00
Daniel Chalef
02efeea8e9
Improve node dedup prompts (#942)
* remove poetry.lock

* Improve dedup prompts

* normalize string formatting in dedupe_nodes.py to use single quotes
2025-09-28 12:18:06 -07:00
Preston Rasmussen
c28bde6b07
fix typo and model selector (#843)
* fix typo and model selector

* bump version
2025-08-18 11:15:45 -04:00
Preston Rasmussen
baa6825708
ensure ascii default to false (#817) 2025-08-08 11:20:02 -04:00
HUGO SON
ce9ef3ca79
Add support for non-ASCII characters in LLM prompts (#805)
* Add support for non-ASCII characters in LLM prompts

- Add ensure_ascii parameter to Graphiti class (default: True)
- Create to_prompt_json helper function for consistent JSON serialization
- Update all prompt files to use new helper function
- Preserve Korean/Japanese/Chinese characters when ensure_ascii=False
- Maintain backward compatibility with existing behavior

Fixes issue where non-ASCII characters were escaped as unicode sequences
in prompts, making them unreadable in LLM logs and potentially affecting
model understanding.

* Remove unused json imports after replacing with to_prompt_json helper

- Fix ruff lint errors (F401) for unused json imports
- All prompt files now use to_prompt_json helper instead of json.dumps
- Maintains clean code style and passes lint checks

* Fix ensure_ascii propagation to all LLM calls

- Add ensure_ascii parameter to maintenance operation functions that were missing it
- Update function signatures in node_operations, community_operations, temporal_operations, and edge_operations
- Ensure all llm_client.generate_response calls receive proper ensure_ascii context
- Fix hardcoded ensure_ascii: True values that prevented non-ASCII character preservation
- Maintain backward compatibility with default ensure_ascii=True
- Complete the fix for issue #804 ensuring Korean/Japanese/Chinese characters are properly handled in LLM prompts
2025-08-08 11:07:32 -04:00
prestonrasmussen
6f3a4f19eb dedupe prompt update 2025-07-29 18:43:42 -04:00
Preston Rasmussen
0675ac2b7d
Bulk ingestion (#698)
* partial

* update

* update

* update

* update

* updates

* updates

* update

* update
2025-07-10 12:14:49 -04:00
Preston Rasmussen
2b27353097
Node name bug fix (#622)
* fixes

* fix bugs

* change version
2025-06-24 17:13:27 -04:00
Preston Rasmussen
760ca7e90c
Node name bug (#605)
* prompt update

* prompt update

* revert quickstart changes
2025-06-18 18:20:28 -04:00
Preston Rasmussen
e8bf81fc6b
add IS_DUPLICATE_OF edges (#599)
* add IS_DUPLICATE_OF edges

* cypher query update

* robust handling
2025-06-17 11:56:55 -04:00
Preston Rasmussen
ebee09b335
Edge extraction and Node Deduplication updates (#564)
* update tests

* updated fact extraction

* optimize node deduplication

* linting

* Update graphiti_core/utils/maintenance/edge_operations.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-06 12:28:52 -04:00
Preston Rasmussen
9422b6f5fb
Node dedupe efficiency (#490)
* update resolve extracted edge

* updated edge resolution

* dedupe nodes update

* single pass node resolution

* updates

* mypy updates

* Update graphiti_core/prompts/dedupe_nodes.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* remove unused imports

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-15 13:56:33 -04:00
Preston Rasmussen
fd9969b5a1
Update dedupe prompt (#457)
* improve dedupe logic

* cut summary length

* update unit tests
2025-05-07 23:23:31 -04:00
Preston Rasmussen
6b85e92105
Fix empty node name issues (#433)
* fixes

* fix

* remove unused imports

* format

* bump version
2025-05-02 12:16:26 -04:00
Preston Rasmussen
2ffc58b3da
small model fix (#432)
* updated dedupe nodes operations

* updates

* Update examples/podcast/podcast_transcript.txt

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* mypy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-02 10:08:25 -04:00
Preston Rasmussen
1193b25fa3
add_episode() refactor (#421)
* temporal updates

* update resolve nodes

* dedupe edge updates

* edge dedupe

* extract attributes

* update dynamic pydantic model

* first pass of extract node attributes

* no errors

* bug fixes

* bug fixes

* prompt updates

* prompt updates

* updates

* updates

* remove unused imports

* update tests based on changes

* remove unused import
2025-04-30 12:08:52 -04:00
Daniel Chalef
7f7a17c926
chore: update dependencies and refactor type hinting (#339)
* Bump version from 0.9.0 to 0.9.1 in pyproject.toml and update google-genai dependency to >=0.1.0

* Bump version from 0.9.1 to 0.9.2 in pyproject.toml

* Update google-genai dependency version to >=0.8.0 in pyproject.toml

* loc file

* Update pyproject.toml to version 0.9.3, restructure dependencies, and modify author format. Remove outdated Google API key note from README.md.

* upgrade poetry and ruff

* Update README.md to include installation instructions for Graphiti with Google Gemini support

* fix to deps since peotry doesn't fully implement PEP 735

* Refactor string formatting in various files to use single quotes for consistency and improve readability. This includes updates in agent.ipynb, quickstart.py, multiple prompt files, and ingest.py and retrieve.py modules.

* Remove optional dependencies from pyproject.toml to streamline project requirements.
2025-04-09 08:05:26 -07:00
Daniel Chalef
0f6ac57dab
chore: update version to 0.9.3 and restructure dependencies (#338)
* Bump version from 0.9.0 to 0.9.1 in pyproject.toml and update google-genai dependency to >=0.1.0

* Bump version from 0.9.1 to 0.9.2 in pyproject.toml

* Update google-genai dependency version to >=0.8.0 in pyproject.toml

* loc file

* Update pyproject.toml to version 0.9.3, restructure dependencies, and modify author format. Remove outdated Google API key note from README.md.

* upgrade poetry and ruff
2025-04-08 20:47:38 -07:00
Preston Rasmussen
7f20b21572
Entity attributes in prompts (#284)
* add node attributes to prompts

* tested

* attribute update
2025-03-04 16:34:19 -05:00
Daniel Chalef
567a8ab74a
Implement OpenAI Structured Output (#225)
* implement so

* bug fixes and typing

* inject schema for non-openai clients

* correct datetime format

* remove List keyword

* Refactor node_operations.py to use updated prompt_library functions

* update example
2024-12-05 07:03:18 -08:00
Preston Rasmussen
a8a73ec38b
Add episode latency improvements (#214)
* reformat prompts

* update prompts

* update

* update

* update

* update

* update

* mypy
2024-11-13 20:13:06 -05:00
Preston Rasmussen
e9e6039b1e
Speed up add episode (#77)
* WIP

* updates

* use uuid for node dedupe

* pret-testing

* parallelized node resolution

* working add_episode

* revert to 4o

* format

* mypy update

* update types
2024-09-03 13:25:52 -04:00
Preston Rasmussen
35a4e5172b
add bulk temporal extraction and improve bulk quality and performance (#67)
* parallelize edge deduping more

* parallelize node insertion more

* improve bulk behavior performance

* dedupe nodes actually works

* add a reranker to search

* bulk dedupe episodes only across the same nodes

* add temporal extraction bulk function

* cleaned up bulk

* default to 4o

* format

* mypy

* mympy

* mypy ignore
2024-08-30 10:48:28 -04:00
Pavlo Paliychuk
0ed7739bc0
Controlled example (#37)
* chore: Add romeo runner

* fix: Linter

* dedupe fixes

* wip

* wip dump

* allbirds

* chore: Update romeo parser

* chore: Anthropic model fix

* allbirds runner

* format

* wip

* mypy updates

* update

* remove r

* update tests

* format

* wip

* wip

* wip

* chore: Strategically update the message

* chore: Add romeo runner

* fix: Linter

* wip

* wip dump

* chore: Update romeo parser

* chore: Anthropic model fix

* wip

* allbirds

* allbirds runner

* format

* wip

* wip

* mypy updates

* update

* remove r

* update tests

* format

* wip

* chore: Strategically update the message

* rebase and fix import issues

* Update package imports for graphiti_core in examples and utils

* nits

* chore: Update OpenAI GPT-4o model to gpt-4o-2024-08-06

* implement groq

* improvments & linting

* cleanup and nits

* Refactor package imports for graphiti_core in examples and utils

* Refactor package imports for graphiti_core in examples and utils

* chore: Nuke unused examples

* chore: Nuke unused examples

* chore: Only run type check on graphiti_core

* fix unit tests

* reformat

* unit test

* fix: Unit tests

* test: Add coverage for extract_date_strings_from_edge

* lint

* remove commented code

---------

Co-authored-by: prestonrasmussen <prasmuss15@gmail.com>
Co-authored-by: Daniel Chalef <131175+danielchalef@users.noreply.github.com>
2024-08-26 10:30:22 -04:00
Daniel Chalef
c5e52153c4
chore: Fix packaging (#38)
* feat: Update project name and description

The project name and description in the `pyproject.toml` file have been updated to reflect the changes made to the project.

* chore: Update pyproject.toml to include core package

The `pyproject.toml` file has been updated to include the `core` package in the list of packages. This change ensures that the `core` package is included when building the project.

* fix imports

* fix importats
2024-08-25 10:07:50 -07:00
Renamed from core/prompts/dedupe_nodes.py (Browse further)