gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4911 commits 5 branches 72 tags 705 MiB
Commit graph

3174 commits

Author SHA1 Message Date
yangdx
c8a17f7ea5 Improve extraction failure log message formatting and consistency 2025-09-11 14:03:21 +08:00
yangdx
7f83a58497 Refactor extraction delimiters from ## to newlines and change tuple delimiter to <|SEP|> 
• Add robust delimiter fixing logic
• Update prompts for single-line format
 2025-09-11 13:44:44 +08:00
yangdx
7fe47fac84 Fix linting 2025-09-10 18:38:21 +08:00
yangdx
db6bba80c9 Log all merges at appropriate level 2025-09-10 18:37:13 +08:00
yangdx
a4bfdb7ddf Fix logging condition to show merges even when no fragments exist if LLM is used 2025-09-10 18:22:10 +08:00
yangdx
02e7462645 feat: enhance LLM output format tolerance for bracket processing 
- Expand bracket tolerance to support additional characters: < > " '
- Implement symmetric handling for both leading and trailing characters
- Replace simple string matching with robust regex-based pattern detection
- Maintain full backward compatibility with existing bracket formats
 2025-09-10 18:10:06 +08:00
yangdx
00de0a4be8 Handle backtick-wrapped brackets in extraction result parsing 
* Support `( and `（ start patterns
* Support )` and ）` end patterns
* Graceful fallback to warning logs
* Strip 2 chars for backtick variants
* Maintain existing bracket logic
 2025-09-10 17:15:03 +08:00
yangdx
19014c6471 feat: enhance entity/relationship merging with description length comparison 
- Implement description length comparison in gleaning merge logic (extract_entities)
- Apply same logic to knowledge graph reconstruction (_rebuild_knowledge_from_chunks)
- Prioritize entities/relationships with longer descriptions for better quality
- Use list() instead of extend() for performance optimization when replacing
 2025-09-10 17:06:57 +08:00
yangdx
e3ebf45a18 Add logging for missing brackets in extraction result processing 2025-09-10 16:10:42 +08:00
yangdx
24242c5bb8 Fix indentation for logging and status updates in merge functions 2025-09-10 15:26:35 +08:00
yangdx
c4506438cd Only log merge messages when there are existing fragments to merge 2025-09-10 15:14:33 +08:00
yangdx
50fddeebbf fix: Remove conversation history from prompt template 
- Delete history section from prompt
- Simplify user query response format
- Remove {history} placeholder variable
 2025-09-10 12:07:34 +08:00
yangdx
a49c8e4a0d Refactor JSON serialization to use newline-separated format 
- Replace json.dumps with line-by-line format
- Apply to entities, relations, text units
- Update truncation key functions
- Maintain ensure_ascii=False setting
- Improve context readability
 2025-09-10 11:59:25 +08:00
yangdx
2dd143c935 Refactor conversation history handling to use LLM native message format 
• Remove get_conversation_turns utility
• Pass history_messages to LLM directly
• Clean up prompt template formatting
 2025-09-10 11:56:58 +08:00
yangdx
e078ab7103 Fix cache handling and context return logic for query parameters 
• Skip cache when only_need_prompt is set
• Update only_need_context condition logic
• Prevent cache bypass in prompt-only mode
 2025-09-10 11:31:48 +08:00
yangdx
6774058670 Merge branch 'main' into tongda/main 2025-09-09 22:43:17 +08:00
yangdx
077d9be5d7 Add Deepseek Style Chain of Thought (CoT) Support for OpenAI Compatible LLM providers 
- Add enable_cot parameter to all LLM APIs
- Implement CoT for OpenAI with <think> tags
- Log warnings for unsupported providers
- Enable CoT in query operations
- Handle streaming and non-streaming CoT
 2025-09-09 22:34:36 +08:00
yangdx
3477e9f919 Merge branch 'main' into tongda/main 2025-09-09 18:27:56 +08:00
yangdx
09abb656b8 Improve log message formatting for better readability 2025-09-09 17:41:09 +08:00
Daniel.y
f064b950fc
Merge pull request #2027 from Matt23-star/main 
Refactor: PostgreSQL
 2025-09-09 15:12:35 +08:00
Daniel.y
92058187f7
Merge pull request #2082 from danielaskdd/prompt-optimization 
Prompt Optimization: remove angle brackets from entity and relationship output formats
 2025-09-09 12:11:39 +08:00
yangdx
564850aa9d Update webui assets and bump api version to 0218 2025-09-09 11:41:02 +08:00
yangdx
06db511f3b Remove angle brackets from entity and relationship output formats 2025-09-09 09:21:23 +08:00
yangdx
6157318408 Update webui assets and bump api to 0217 2025-09-08 23:37:34 +08:00
yangdx
3059089e7d Fix logging order in pipeline history trimming 2025-09-08 23:00:44 +08:00
yangdx
8d53ef7ff0 Increase default Gunicorn worker timeout from 210 to 300 seconds 2025-09-08 20:03:21 +08:00
yangdx
56df4ac4b1 Update webui assets and bump api version to 0216 2025-09-08 19:36:49 +08:00
yangdx
de4fe8bc7d Improve uvicorn workers warning message clarity 2025-09-08 16:05:51 +08:00
yangdx
9437df83cc Add memory management for pipeline history messages 
- Trim history at 10k messages
- Keep latest 5k messages
- Prevent memory growth
- Add logging for trim events
 2025-09-08 15:56:35 +08:00
yangdx
d218f15a62 Refactor entity extraction with system prompts and output limits 
- Add system/user prompt separation
- Set max tokens for endless output fix
- Improve extraction error logging
- Update cache type from extract to summary
 2025-09-08 15:20:45 +08:00
yangdx
451f488f72 Add debug logging for client configs in OpenAI LLM function 2025-09-07 02:29:37 +08:00
yangdx
c87eb2cfcf Increase timeout buffers for async function calls 
• Extend execution timeout buffer to 150s
• Extend task duration buffer to 180s
• Account for low-level retry delays
• Improve health check phase handling
• Reduce timeout-related failures
 2025-09-06 23:56:24 +08:00
yangdx
725db3b240 Fix linting in the prompt 2025-09-06 11:16:49 +08:00
yangdx
219a08b7c9 Restore completion_delimiter 2025-09-06 11:13:37 +08:00
yangdx
528d04a0e4 Update prompt template delimiters 2025-09-06 10:35:06 +08:00
yangdx
5446815008 Refactor entity extraction prompts and remove completion delimiter. 
- Remove `completion_delimiter` from prompts
- Update input/output format markers
 2025-09-06 09:13:51 +08:00
yangdx
0a62f02e84 Improve edge logging format and exception prefixes 2025-09-06 08:35:52 +08:00
yangdx
387d817fc2 Remove trailing colons from queue names in function wrappers 2025-09-06 00:53:05 +08:00
yangdx
e4dd83ee3d Change tooltips font to a smaller one 2025-09-05 22:14:28 +08:00
yangdx
c77e045fb3 Update webui assets and bump api version to 0215 2025-09-05 22:05:52 +08:00
yangdx
6be462511f Add error prefixing for better debugging context in async operations 
* Add create_prefixed_exception utility
* Prefix entity processing errors
* Prefix relationship processing errors
* Prefix chunk extraction progress info
* Maintain original exception chains
 2025-09-05 21:28:00 +08:00
yangdx
385668dec5 Fix malformed tuple delimiters in extraction result processing 2025-09-05 17:14:42 +08:00
yangdx
a1df76a4ea Optimize LLM/embedding config caching to reduce repeated parsing overhead 
• Add LLMConfigCache class for smart caching
• Pre-process OpenAI/Ollama configurations
• Create optimized function factories
• Reduce redundant option parsing calls
 2025-09-05 16:36:08 +08:00
yangdx
17d665c9f3 Limit history messages to latest 1000 entries with truncation indicator 
• Limit history to 1000 latest messages
• Add truncation message when needed
• Show count of truncated messages
• Update API documentation
• Prevent memory issues with large logs
 2025-09-05 12:31:36 +08:00
yangdx
be3f0ebbe5 Simplify entity extraction prompt instructions and remove delimiter 2025-09-04 23:42:11 +08:00
yangdx
3f56c6820c Reorder language and completion delimiter instructions in prompt 2025-09-04 23:05:16 +08:00
yangdx
2c551cb5db Add support for Chinese book title marks in normalize_extracted_info 2025-09-04 18:51:57 +08:00
yangdx
f19cce16be Fix incorrect variable name in NetworkXStorage file path 
- Fix working_dir -> workspace_dir typo
- Correct GraphML file path generation
 2025-09-04 18:31:53 +08:00
yangdx
50adf64fab Fix linting in prompt 2025-09-04 15:22:36 +08:00
yangdx
94114df995 Improve prompt clarity and structure 2025-09-04 14:53:27 +08:00