gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5431 commits 5 branches 72 tags 705 MiB
Commit graph

603 commits

Author SHA1 Message Date
yangdx
3fa79026e0 Fix Entity Source IDs Tracking Problem 
- Handle existing node updates properly in edge merging stage
- Fix source_ids merging logic
- Reorder entity deletion and optimize node operations
- Delete relationships before entities
- Add edge existence debugging logs
 2025-10-29 01:19:55 +08:00
yangdx
c81a56a113 Fix entity and relationship deletion when no chunk references remain 2025-10-28 16:02:35 +08:00
yangdx
5155edd8d2 feat: Improve entity merge and edit UX 
- **API:** The `graph/entity/edit` endpoint now returns a detailed `operation_summary` for better client-side handling of update, rename, and merge outcomes.
- **Web UI:** Added an "auto-merge on rename" option. The UI now gracefully handles merge success, partial failures (update OK, merge fail), and other errors with specific user feedback.
 2025-10-27 23:42:08 +08:00
yangdx
2c09adb8d3 Add chunk tracking support to entity merge functionality 
- Pass chunk storages to merge function
- Merge relation chunk tracking data
- Merge entity chunk tracking data
- Delete old chunk tracking records
- Persist chunk storage updates
 2025-10-27 02:06:21 +08:00
yangdx
3fbd704bf9 Enhance entity/relation editing with chunk tracking synchronization 
• Add chunk storage sync to edit ops
• Implement incremental chunk ID updates
• Support entity renaming migrations
• Normalize relation keys consistently
• Preserve chunk references on edits
 2025-10-26 14:34:56 +08:00
yangdx
29bf593663 Fix entity and relation chunk cleanup in deletion pipeline 
• Delete from entity_chunks storage
• Delete from relation_chunks storage
 2025-10-25 22:32:27 +08:00
yangdx
a9bc348446 Remove enable_logging parameter from data init lock call 2025-10-25 11:48:14 +08:00
yangdx
97a2ee4ef1 Rename rebuild function name and improve relationship logging format 2025-10-25 11:17:43 +08:00
yangdx
a9ec15e669 Resolve lock leakage issue during user cancellation handling 
• Change default log level to INFO
• Force enable error logging output
• Add lock cleanup rollback protection
• Handle LLM cache persistence errors
• Fix async task exception handling
 2025-10-25 03:06:45 +08:00
yangdx
77336e50b6 Improve error handling and add cancellation checks in pipeline 2025-10-24 17:54:17 +08:00
yangdx
743aefc655 Add pipeline cancellation feature for graceful processing termination 
• Add cancel_pipeline API endpoint
• Implement PipelineCancelledException
• Add cancellation checks in main loop
• Handle task cancellation gracefully
• Mark cancelled docs as FAILED
 2025-10-24 14:08:12 +08:00
yangdx
b76350a3bc Fix linting 2025-10-22 12:53:42 +08:00
yangdx
d7e2527e1a Handle cache deletion errors gracefully instead of raising exceptions 2025-10-22 12:53:19 +08:00
yangdx
162370b6e6 Add optional LLM cache deletion when deleting documents 
• Add delete_llm_cache parameter to API
• Collect cache IDs from text chunks
• Delete cache after graph operations
• Update UI with new checkbox option
• Add i18n translations for cache option
 2025-10-22 12:19:23 +08:00
yangdx
e5e16b7bd1 Fix Redis data migration error 
• Use proper Redis connection context
• Fix namespace pattern for key scanning
• Propagate storage check exceptions
• Remove defensive error swallowing
 2025-10-21 16:27:04 +08:00
yangdx
a9fec26798 Add file path limit configuration for entities and relations 
• Add MAX_FILE_PATHS env variable
• Implement file path count limiting
• Support KEEP/FIFO strategies
• Add truncation placeholder
• Remove old build_file_path function
 2025-10-20 20:12:53 +08:00
yangdx
dc62c78f98 Add entity/relation chunk tracking with configurable source ID limits 
- Add entity_chunks & relation_chunks storage
- Implement KEEP/FIFO limit strategies
- Update env.example with new settings
- Add migration for chunk tracking data
- Support all KV storage
 2025-10-20 15:24:15 +08:00
yangdx
9f49e56a44 Merge branch 'main' into feat-entity-size-caps 2025-10-17 15:59:44 +08:00
DivinesLight
c06522b927 Get max source Id config from .env and lightRAG init 2025-10-15 18:24:38 +05:00
yangdx
29bac49fb9 Handle empty query results by returning None instead of fail responses 
• Return None when no context found
• Add structured failure metadata
• Use PROMPTS["fail_response"] for content
• Keep API compatible
 2025-10-15 12:04:49 +08:00
yangdx
130b4959dc Add PREPROCESSED (multimodal_processed) status for multimodal document processing 
• Add DocStatus.PREPROCESSED enum value
• Update API routes and response models
• Add preprocessed filter in web UI
• Update localization files
• Handle preprocessed status in deletion
 2025-10-14 14:02:05 +08:00
yangdx
074f0c8b23 Update docstring for adelete_by_doc_id method clarity 2025-10-12 10:12:45 +08:00
yangdx
457d51952e Add doc_name field to full docs storage 
- Store file_path in full_docs storage
- Update PostgreSQL implementation by map file_path to doc_name
- Other storage implementation automatically handles the new field
 2025-10-05 11:44:27 +08:00
yangdx
1766cddd6c Fix mode parameter serialization error in Ollama chat API 
• Use mode.value for API requests
• Add debug logging in aquery_llm
 2025-09-27 15:11:51 +08:00
yangdx
8cd4139cbf refactor: fix double query problem by add aquery_llm function for consistent response handling 
- Add new aquery_llm/query_llm methods providing structured responses
- Consolidate /query and /query/stream endpoints to use unified aquery_llm
- Optimize cache handling by moving cache checks before LLM calls
 2025-09-26 19:05:03 +08:00
yangdx
b848ca49e6 Fix linting 2025-09-25 16:22:00 +08:00
yangdx
b08b8a6a6a Add reference list support to query API endpoints with unified result handling 
• Add include_references param to QueryRequest
• Extend QueryResponse with references field
• Create unified QueryResult data structures
• Refactor kg_query and naive_query functions
• Update streaming to send references first
 2025-09-25 16:21:42 +08:00
yangdx
5eb4a4b799 feat: simplify citations, add reference merging, and restructure API response format 2025-09-24 14:30:10 +08:00
yangdx
c0d5abba6b Fix linting 2025-09-15 02:59:21 +08:00
yangdx
b1c8206346 Add aquery_data endpoint for structured retrieval without LLM generation 
- Add QueryDataResponse model
- Implement /query/data endpoint
- Add aquery_data method to LightRAG
- Return entities, relationships, chunks
 2025-09-15 02:15:14 +08:00
yangdx
82a67354d0 Code formatting improvements and style consistency fixes 
* Remove trailing whitespace
* Fix function signature ellipsis style
 2025-09-14 17:49:02 +08:00
yangdx
0ffb5d5f2d Replace search API with aquery_data for consistent raw data retrieval, mirroring aquery results 
• Reuse existing query logic paths and remove kg_search function entirely
• Update kg_query/naive_query to return raw data as needed
 2025-09-13 15:30:29 +08:00
yangdx
6774058670 Merge branch 'main' into tongda/main 2025-09-09 22:43:17 +08:00
yangdx
077d9be5d7 Add Deepseek Style Chain of Thought (CoT) Support for OpenAI Compatible LLM providers 
- Add enable_cot parameter to all LLM APIs
- Implement CoT for OpenAI with <think> tags
- Log warnings for unsupported providers
- Enable CoT in query operations
- Handle streaming and non-streaming CoT
 2025-09-09 22:34:36 +08:00
yangdx
3477e9f919 Merge branch 'main' into tongda/main 2025-09-09 18:27:56 +08:00
yangdx
3059089e7d Fix logging order in pipeline history trimming 2025-09-08 23:00:44 +08:00
yangdx
9437df83cc Add memory management for pipeline history messages 
- Trim history at 10k messages
- Keep latest 5k messages
- Prevent memory growth
- Add logging for trim events
 2025-09-08 15:56:35 +08:00
yangdx
387d817fc2 Remove trailing colons from queue names in function wrappers 2025-09-06 00:53:05 +08:00
yangdx
de972f6222 Rename method for clarity and improve code readability 
- Rename _process_entity_relation_graph to _process_extract_entities
 2025-09-04 11:48:31 +08:00
Tong Da
dc7ce98c7e Add search interface to lightrag. 2025-09-01 02:40:40 +08:00
yangdx
1a015a7015 Add queue_name parameter to priority_limit_async_func_call for better logging 
• Add queue_name parameter to decorator
• Update all log messages with queue names
• Pass specific names for LLM and embedding
 2025-08-31 23:47:22 +08:00
yangdx
925e631a9a refac: Add robust time out handling for LLM request 2025-08-29 13:50:35 +08:00
yangdx
ff0a18e08c Unify SUMMARY_LANGUANGE and ENTITY_TYPES implementation method 2025-08-27 12:23:22 +08:00
Thibo Rosemplatt
c3aabfc251 Merge branch 'main' into entityTypesServerSupport 2025-08-26 21:48:20 +02:00
yangdx
d3623cc9ae fix: resolve infinite loop risk in _handle_entity_relation_summary 
- Ensure oversized descriptions are force-merged with subsequent ones
- Add len(current_list) <= 2 termination condition to guarantee convergence
- Implement token-based truncation in _summarize_descriptions to prevent overflow
 2025-08-26 21:58:31 +08:00
yangdx
6bcfe696ee feat: add output length recommendation and description type to LLM summary 
- Add SUMMARY_LENGTH_RECOMMENDED parameter (600 tokens)
- Optimize prompt temple for LLM summary
 2025-08-26 14:41:12 +08:00
yangdx
cb0fe38b9a Fix linting 2025-08-26 02:22:34 +08:00
yangdx
de2daf6565 refac: Rename summary_max_tokens to summary_context_size, comprehensive parameter validation for summary configuration 
- Update algorithm logic in operate.py for better token management
- Fix health endpoint to use correct parameter names
 2025-08-26 01:35:50 +08:00
yangdx
0b1b264a5d refactor: optimize graph lock scope in document deletion 
- Move dependency analysis outside graph database lock
- Add persistence call before lock release to prevent dirty reads
 2025-08-25 17:46:32 +08:00
Thibo Rosemplatt
d054ec5d00 Added entity_types as a user defined variable (via .env) 2025-08-23 20:16:11 +02:00