gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5584 commits 5 branches 72 tags 705 MiB
Commit graph

3568 commits

Author SHA1 Message Date
yangdx
e95b02fb55 Refactor storage selection UI with dynamic numbering and inline prompts 
• Remove standalone get_user_choice method
• Add dynamic sequential numbering
• Inline choice validation logic
• Remove redundant storage type prints
• Improve excluded storage handling
 2025-11-08 20:42:27 +08:00
yangdx
5be04263b2 Fix deadlock in JSON cache migration and prevent same storage selection 
- Snapshot JSON data before yielding batches
- Release lock during batch processing
- Exclude source type from target selection
- Add detailed docstring for lock behavior
- Filter available storage types properly
 2025-11-08 19:58:36 +08:00
yangdx
6b9f13c792 Enhance LLM cache migration tool with streaming and improved UX 
- Add streaming migration for memory efficiency
- Implement graceful exit with Enter/0
- Add progress indicators for counting
- Optimize batch processing by storage type
- Update docs with new progress displays
 2025-11-08 19:38:00 +08:00
yangdx
d0d31e9262 Improve LLM cache migration tool configuration and messaging 2025-11-08 18:52:33 +08:00
yangdx
6fc54d3625 Move LLM cache migration tool to lightrag.tools module 
- Relocated tool to proper package structure
- Updated import paths and documentation
- Added shared storage initialization
- Fixed module path resolution
- Updated usage instructions
 2025-11-08 18:33:13 +08:00
yangdx
cf732dbfc6 Bump core version to 1.4.9.9 and API to 0252 2025-11-08 11:27:26 +08:00
yangdx
a624a9508a Add Gemini to APIs requiring embedding dimension parameter 2025-11-08 03:54:50 +08:00
yangdx
de4ed73652 Add Gemini embedding support 
- Implement gemini_embed function
- Add gemini to embedding binding choices
- Add L2 normalization for dims < 3072
 2025-11-08 03:34:30 +08:00
yangdx
f83ea3394e Add section header comment for Gemini binding options 2025-11-08 02:07:31 +08:00
yangdx
0b2a15c452 Centralize embedding_send_dim config through args instead of env var 2025-11-08 01:52:23 +08:00
yangdx
03cc6262c4 Prohibit direct access to internal functions of EmbeddingFunc. 
• Fix similarity search error in query stage
• Remove redundant null checks
• Improve log readability
 2025-11-08 01:43:36 +08:00
yangdx
ffeeae4208 refactor: simplify jina embedding dimension handling 2025-11-07 22:09:57 +08:00
yangdx
01b07b2be5 Refactor Jina embedding dimension by changing param to optional with default 2025-11-07 22:04:34 +08:00
yangdx
d95efcb9ad Fix linting 2025-11-07 21:27:54 +08:00
yangdx
ce28f30ca6 Add embedding_dim parameter support to embedding functions 
• Pass embedding_dim to jina_embed call
• Pass embedding_dim to openai_embed call
 2025-11-07 21:23:59 +08:00
yangdx
c14f25b7f8 Add mandatory dimension parameter handling for Jina API compliance 2025-11-07 21:08:34 +08:00
yangdx
d8a6355e41 Merge branch 'main' into apply-dim-to-embedding-call 2025-11-07 20:48:22 +08:00
yangdx
33a1482f7f Add optional embedding dimension parameter control via env var 
* Add EMBEDDING_SEND_DIM environment variable
* Update Jina/OpenAI embed functions
* Add send_dimensions to EmbeddingFunc
* Auto-inject embedding_dim when enabled
* Add parameter validation warnings
 2025-11-07 20:46:40 +08:00
yangdx
fc40a36968 Add timeout support to Gemini LLM and improve parameter handling 
• Add timeout parameter to Gemini client
• Convert timeout seconds to milliseconds
• Update function signatures consistently
• Add Gemini thinking config example
• Clean up parameter documentation
 2025-11-07 15:50:14 +08:00
yangdx
3cb4eae492 Add Chain of Thought support to Gemini LLM integration 
- Extract thoughts from response parts
- Add COT enable/disable parameter
 2025-11-07 15:22:14 +08:00
yangdx
6686edfd35 Update Gemini LLM options: add seed and thinking config, remove MIME type 2025-11-07 14:32:42 +08:00
Yasiru Rangana
d94aae9c5e Add dimensions parameter support to openai_embed() 2025-11-07 09:55:06 +11:00
yangdx
8c27555358 Fix Gemini response parsing to avoid warnings from non-text parts 2025-11-07 04:00:37 +08:00
yangdx
ea141e2779 Fix: Remove redundant entity/relation chunk deletions 2025-11-07 02:56:16 +08:00
yangdx
5bcd2926ca Bump API version to 0251 2025-11-06 21:45:47 +08:00
yangdx
04ed709b34 Optimize entity deletion by batching edge queries to avoid N+1 problem 
• Add batch get_nodes_edges_batch call
• Remove individual get_node_edges calls
• Improve query performance
 2025-11-06 21:34:47 +08:00
yangdx
3276b7a49d Fix linting 2025-11-06 20:48:51 +08:00
yangdx
155f59759b Fix node ID normalization and improve batch operation consistency 
• Remove premature ID normalization
• Add lookup mapping for node resolution
• Filter results by requested nodes only
• Improve error logging with workspace
 2025-11-06 20:34:53 +08:00
yangdx
807d2461d3 Remove unused chunk-based node/edge retrieval methods 2025-11-06 18:17:10 +08:00
yangdx
831e658ed8 Update readme 2025-11-06 16:26:07 +08:00
yangdx
6e36ff41e1 Fix linting 2025-11-06 16:01:24 +08:00
yangdx
5f49cee20f Merge branch 'main' into VOXWAVE-FOUNDRY/main 2025-11-06 15:37:35 +08:00
yangdx
9c05706062 Add separate endpoint configuration for LLM and embeddings in evaluation 
- Split LLM and embedding API configs
- Add fallback chain for API keys
- Update docs with usage examples
 2025-11-05 18:54:38 +08:00
yangdx
994a82dc7f Suppress token usage warnings for custom OpenAI-compatible endpoints 
• Add warning filter for token usage
• Support vLLM, SGLang endpoints
• Non-critical for RAGAS evaluation
 2025-11-05 18:25:28 +08:00
yangdx
f490622b72 Doc: Refactor evaluation README to improve clarity and structure 2025-11-05 10:43:55 +08:00
yangdx
a73314a4ba Refactor evaluation results display and logging format 2025-11-05 10:08:17 +08:00
yangdx
06b91d00f8 Improve RAG evaluation progress eval index display with zero padding 2025-11-05 09:46:07 +08:00
yangdx
2823f92fb6 Fix tqdm progress bar conflicts in concurrent RAG evaluation 
• Add position pool for tqdm bars
• Serialize tqdm creation with lock
• Set leave=False to clear completed bars
• Pass position/lock to eval tasks
• Import tqdm.auto for better display
 2025-11-05 02:04:13 +08:00
yangdx
e5abe9dd3d Restructure semaphore control to manage entire evaluation pipeline 
• Move rag_semaphore to wrap full function
• Increase RAG concurrency to 2x eval limit
• Prevent memory buildup from slow evals
• Keep eval_semaphore for RAGAS control
 2025-11-05 01:07:53 +08:00
yangdx
83715a3ac1 Implement two-stage pipeline for RAG evaluation with separate semaphores 
• Split RAG gen and eval stages
• Add rag_semaphore for stage 1
• Add eval_semaphore for stage 2
• Improve concurrency control
• Update connection pool limits
 2025-11-05 00:36:09 +08:00
yangdx
d36be1f499 Improve RAGAS evaluation progress tracking and clean up output handling 
• Add tqdm progress bar for eval steps
• Pass progress bar to RAGAS evaluate
• Ensure progress bar cleanup in finally
• Remove redundant output buffer flushes
 2025-11-05 00:16:02 +08:00
yangdx
c358f405a9 Update evaluation defaults and expand sample dataset 
• Lower concurrent evals from 3 to 2
• Standardize project names in samples
• Add 3 new evaluation questions
• Expand ground truth detail coverage
• Improve dataset comprehensiveness
 2025-11-04 22:17:17 +08:00
yangdx
41c26a3677 feat: add command-line args to RAG evaluation script 
- Add --dataset and --ragendpoint flags
- Support short forms -d and -r
- Update README with usage examples
 2025-11-04 21:40:27 +08:00
yangdx
d4b8a229b9 Update RAGAS evaluation to use gpt-4o-mini and improve compatibility 
- Change default model to gpt-4o-mini
- Add deprecation warning suppression
- Update docs and comments for LightRAG
- Improve output formatting and timing
 2025-11-04 18:50:53 +08:00
yangdx
6d61f70b92 Clean up RAG evaluator logging and remove excessive separator lines 
• Remove excessive separator lines
• Add RAGAS concurrency comment
• Fix output buffer timing
 2025-11-04 18:04:19 +08:00
yangdx
4e4b8d7e25 Update RAG evaluation metrics to use class instances instead of objects 
• Import metric classes not instances
• Instantiate metrics with () syntax
 2025-11-04 15:56:57 +08:00
yangdx
7abc687742 Add comprehensive configuration and compatibility fixes for RAGAS 
- Fix RAGAS LLM wrapper compatibility
- Add concurrency control for rate limits
- Add eval env vars for model config
- Improve error handling and logging
- Update documentation with examples
 2025-11-04 14:39:27 +08:00
yangdx
72db042667 Update .env loading and add API authentication to RAG evaluator 
• Load .env from current directory
• Support LIGHTRAG_API_KEY auth header
• Override=False for env precedence
• Add Bearer token to API requests
• Enable per-instance .env configs
 2025-11-04 10:59:09 +08:00
anouarbm
ad2d3c2cc0 Merge remote-tracking branch 'origin/main' into feat/ragas-evaluation 2025-11-03 13:48:14 +01:00
anouarbm
debfa0ec96 Merge branch 'feat/ragas-evaluation' of https://github.com/anouar-bm/LightRAG into feat/ragas-evaluation 2025-11-03 13:30:16 +01:00