gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5597 commits 5 branches 72 tags 705 MiB
Commit graph

3576 commits

Author SHA1 Message Date
yangdx
7bc6ccea19 Add uv package manager support to installation docs 2025-11-09 04:31:07 +08:00
yangdx
754d2ad297 Add documentation for LLM cache migration between storage types 2025-11-09 00:41:07 +08:00
yangdx
a75efb06dc Fix: prevent source data corruption by target upsert function 
• Prevent mutations bugs by using copy() when storing cache values
• Protect filtered cache data and ensure batch data isolation
 2025-11-09 00:02:19 +08:00
yangdx
987bc09cab Update LLM cache migration docs and improve UX prompts 2025-11-08 23:48:19 +08:00
yangdx
1a91bcdb5f Improve storage config validation and add config.ini fallback support 
• Add MongoDB env requirements
• Support config.ini fallback
• Warn on missing env vars
• Check available storage count
• Show config source info
 2025-11-08 22:48:49 +08:00
yangdx
57ee7d5ac8 Merge branch 'main' into llm-cache-migrate 2025-11-08 22:15:46 +08:00
yangdx
3d9de5ed03 feat: improve Gemini client error handling and retry logic 
• Add google-api-core dependency
• Add specific exception handling
• Create InvalidResponseError class
• Update retry decorators
• Fix empty response handling
 2025-11-08 22:10:09 +08:00
yangdx
1864b28242 Add colored output formatting to migration confirmation display 2025-11-08 21:16:41 +08:00
yangdx
e95b02fb55 Refactor storage selection UI with dynamic numbering and inline prompts 
• Remove standalone get_user_choice method
• Add dynamic sequential numbering
• Inline choice validation logic
• Remove redundant storage type prints
• Improve excluded storage handling
 2025-11-08 20:42:27 +08:00
yangdx
5be04263b2 Fix deadlock in JSON cache migration and prevent same storage selection 
- Snapshot JSON data before yielding batches
- Release lock during batch processing
- Exclude source type from target selection
- Add detailed docstring for lock behavior
- Filter available storage types properly
 2025-11-08 19:58:36 +08:00
yangdx
6b9f13c792 Enhance LLM cache migration tool with streaming and improved UX 
- Add streaming migration for memory efficiency
- Implement graceful exit with Enter/0
- Add progress indicators for counting
- Optimize batch processing by storage type
- Update docs with new progress displays
 2025-11-08 19:38:00 +08:00
yangdx
d0d31e9262 Improve LLM cache migration tool configuration and messaging 2025-11-08 18:52:33 +08:00
yangdx
6fc54d3625 Move LLM cache migration tool to lightrag.tools module 
- Relocated tool to proper package structure
- Updated import paths and documentation
- Added shared storage initialization
- Fixed module path resolution
- Updated usage instructions
 2025-11-08 18:33:13 +08:00
yangdx
cf732dbfc6 Bump core version to 1.4.9.9 and API to 0252 2025-11-08 11:27:26 +08:00
yangdx
a624a9508a Add Gemini to APIs requiring embedding dimension parameter 2025-11-08 03:54:50 +08:00
yangdx
de4ed73652 Add Gemini embedding support 
- Implement gemini_embed function
- Add gemini to embedding binding choices
- Add L2 normalization for dims < 3072
 2025-11-08 03:34:30 +08:00
yangdx
f83ea3394e Add section header comment for Gemini binding options 2025-11-08 02:07:31 +08:00
yangdx
0b2a15c452 Centralize embedding_send_dim config through args instead of env var 2025-11-08 01:52:23 +08:00
yangdx
03cc6262c4 Prohibit direct access to internal functions of EmbeddingFunc. 
• Fix similarity search error in query stage
• Remove redundant null checks
• Improve log readability
 2025-11-08 01:43:36 +08:00
yangdx
ffeeae4208 refactor: simplify jina embedding dimension handling 2025-11-07 22:09:57 +08:00
yangdx
01b07b2be5 Refactor Jina embedding dimension by changing param to optional with default 2025-11-07 22:04:34 +08:00
yangdx
d95efcb9ad Fix linting 2025-11-07 21:27:54 +08:00
yangdx
ce28f30ca6 Add embedding_dim parameter support to embedding functions 
• Pass embedding_dim to jina_embed call
• Pass embedding_dim to openai_embed call
 2025-11-07 21:23:59 +08:00
yangdx
c14f25b7f8 Add mandatory dimension parameter handling for Jina API compliance 2025-11-07 21:08:34 +08:00
yangdx
d8a6355e41 Merge branch 'main' into apply-dim-to-embedding-call 2025-11-07 20:48:22 +08:00
yangdx
33a1482f7f Add optional embedding dimension parameter control via env var 
* Add EMBEDDING_SEND_DIM environment variable
* Update Jina/OpenAI embed functions
* Add send_dimensions to EmbeddingFunc
* Auto-inject embedding_dim when enabled
* Add parameter validation warnings
 2025-11-07 20:46:40 +08:00
yangdx
fc40a36968 Add timeout support to Gemini LLM and improve parameter handling 
• Add timeout parameter to Gemini client
• Convert timeout seconds to milliseconds
• Update function signatures consistently
• Add Gemini thinking config example
• Clean up parameter documentation
 2025-11-07 15:50:14 +08:00
yangdx
3cb4eae492 Add Chain of Thought support to Gemini LLM integration 
- Extract thoughts from response parts
- Add COT enable/disable parameter
 2025-11-07 15:22:14 +08:00
yangdx
6686edfd35 Update Gemini LLM options: add seed and thinking config, remove MIME type 2025-11-07 14:32:42 +08:00
Yasiru Rangana
d94aae9c5e Add dimensions parameter support to openai_embed() 2025-11-07 09:55:06 +11:00
yangdx
8c27555358 Fix Gemini response parsing to avoid warnings from non-text parts 2025-11-07 04:00:37 +08:00
yangdx
ea141e2779 Fix: Remove redundant entity/relation chunk deletions 2025-11-07 02:56:16 +08:00
yangdx
5bcd2926ca Bump API version to 0251 2025-11-06 21:45:47 +08:00
yangdx
04ed709b34 Optimize entity deletion by batching edge queries to avoid N+1 problem 
• Add batch get_nodes_edges_batch call
• Remove individual get_node_edges calls
• Improve query performance
 2025-11-06 21:34:47 +08:00
yangdx
3276b7a49d Fix linting 2025-11-06 20:48:51 +08:00
yangdx
155f59759b Fix node ID normalization and improve batch operation consistency 
• Remove premature ID normalization
• Add lookup mapping for node resolution
• Filter results by requested nodes only
• Improve error logging with workspace
 2025-11-06 20:34:53 +08:00
yangdx
807d2461d3 Remove unused chunk-based node/edge retrieval methods 2025-11-06 18:17:10 +08:00
yangdx
831e658ed8 Update readme 2025-11-06 16:26:07 +08:00
yangdx
6e36ff41e1 Fix linting 2025-11-06 16:01:24 +08:00
yangdx
5f49cee20f Merge branch 'main' into VOXWAVE-FOUNDRY/main 2025-11-06 15:37:35 +08:00
yangdx
9c05706062 Add separate endpoint configuration for LLM and embeddings in evaluation 
- Split LLM and embedding API configs
- Add fallback chain for API keys
- Update docs with usage examples
 2025-11-05 18:54:38 +08:00
yangdx
994a82dc7f Suppress token usage warnings for custom OpenAI-compatible endpoints 
• Add warning filter for token usage
• Support vLLM, SGLang endpoints
• Non-critical for RAGAS evaluation
 2025-11-05 18:25:28 +08:00
yangdx
f490622b72 Doc: Refactor evaluation README to improve clarity and structure 2025-11-05 10:43:55 +08:00
yangdx
a73314a4ba Refactor evaluation results display and logging format 2025-11-05 10:08:17 +08:00
yangdx
06b91d00f8 Improve RAG evaluation progress eval index display with zero padding 2025-11-05 09:46:07 +08:00
yangdx
2823f92fb6 Fix tqdm progress bar conflicts in concurrent RAG evaluation 
• Add position pool for tqdm bars
• Serialize tqdm creation with lock
• Set leave=False to clear completed bars
• Pass position/lock to eval tasks
• Import tqdm.auto for better display
 2025-11-05 02:04:13 +08:00
yangdx
e5abe9dd3d Restructure semaphore control to manage entire evaluation pipeline 
• Move rag_semaphore to wrap full function
• Increase RAG concurrency to 2x eval limit
• Prevent memory buildup from slow evals
• Keep eval_semaphore for RAGAS control
 2025-11-05 01:07:53 +08:00
yangdx
83715a3ac1 Implement two-stage pipeline for RAG evaluation with separate semaphores 
• Split RAG gen and eval stages
• Add rag_semaphore for stage 1
• Add eval_semaphore for stage 2
• Improve concurrency control
• Update connection pool limits
 2025-11-05 00:36:09 +08:00
yangdx
d36be1f499 Improve RAGAS evaluation progress tracking and clean up output handling 
• Add tqdm progress bar for eval steps
• Pass progress bar to RAGAS evaluate
• Ensure progress bar cleanup in finally
• Remove redundant output buffer flushes
 2025-11-05 00:16:02 +08:00
yangdx
c358f405a9 Update evaluation defaults and expand sample dataset 
• Lower concurrent evals from 3 to 2
• Standardize project names in samples
• Add 3 new evaluation questions
• Expand ground truth detail coverage
• Improve dataset comprehensiveness
 2025-11-04 22:17:17 +08:00