gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4529 commits 5 branches 72 tags 705 MiB
Commit graph

4529 commits

This branch
This branch
All branches
Author SHA1 Message Date
yangdx
44204abef7 Fix linting 2025-08-10 10:59:32 +08:00
yangdx
eb2320e556 Fix: Initialize first_stage_tasks and entity_relation_task to prevent empty-task cancel errors 
- Initialize first_stage_tasks = [] and entity_relation_task = None at coroutine start
- Ensure cancel block safely handles no-op when tasks lists are empty
 2025-08-10 10:45:41 +08:00
yangdx
ffb642a5ce Fix linting 2025-08-09 08:41:41 +08:00
yangdx
ecd7777e61 Update OpenAI embedding handling for both list and base64 embeddings 
- Fix OpenAI embedding array parsing
- Improve embedding data type safety
 2025-08-09 08:40:33 +08:00
yangdx
cf064579ce Remove deprecated keyword extraction query methods 
- Delete query_with_keywords function
- Remove kg_query_with_keywords helper
- Drop separate keyword extraction methods
 2025-08-08 14:59:39 +08:00
yangdx
f5ac6a9f4b Add default Ollama embedding context length 
- Set default context length to 8192
- Overide the default context lenght for LLM in binding_options.py
 2025-08-08 13:51:25 +08:00
yangdx
c2eefec707 Merge branch 'postgres-vector-index' 2025-08-08 03:01:34 +08:00
yangdx
16c9a81f4c feat: support config.ini for PostgreSQL vector index settings 
- Add support for reading vector_index_type, hnsw_m, hnsw_ef, and ivfflat_lists from config.ini
- Maintain backward compatibility with environment variables
- Update config.ini.example with new PostgreSQL vector index options
- Follow existing configuration priority: env vars > config.ini > defaults
 2025-08-08 02:55:49 +08:00
yangdx
dec4148075 Merge branch 'main' into Matt23-star/main 2025-08-08 02:24:34 +08:00
yangdx
f38e10559e Update PostgreSQL vector index configuration 
- Remove FLAT index support
- Standardize on HNSW as default
- Add dimension validation
- Improve error logging
- Clean up index creation code
 2025-08-08 02:21:06 +08:00
Daniel.y
2f289f6e25
Merge pull request #1924 from danielaskdd/neo4j-connection-lifetime 
Refact：Enhanced Neo4j Connection Lifecycle Management
 2025-08-08 01:16:42 +08:00
yangdx
f4ef254de2 fix(neo4j): enhance connection lifecycle management to prevent timeout errors 
- Add max_connection_lifetime, liveness_check_timeout, keep_alive parameters
- Extend retry mechanisms for connection reset scenarios
- Update config examples with new Neo4j connection options
- Resolves ClientTimeoutException during data insertion operations
 2025-08-08 01:07:45 +08:00
Daniel.y
c8a44f5657
Merge pull request #1923 from danielaskdd/fix-context-format 
Fix: Unify  document chunks context format in only_need_context query
 2025-08-08 00:05:26 +08:00
yangdx
eded6d1187 Unify document chunks context format in only_need_context query 
- Update Document Chunks label to include (DC) abbreviation
 2025-08-08 00:02:53 +08:00
Matt23-star
727ca43d3c feat: add vector index creation functionality for PostgreSQL 2025-08-07 23:07:18 +08:00
yangdx
7780776af6 Update env.example 2025-08-06 18:50:58 +08:00
Daniel.y
a6ef29cef6
Merge pull request #1915 from danielaskdd/optimize-llm-cache 
Refact: Optimized LLM Cache Hash Key Generation by Including All Query Parameters
 2025-08-06 01:04:02 +08:00
yangdx
2dab4e321d Bump api version to 0199 2025-08-06 01:03:35 +08:00
yangdx
a04c11a598 Remove deprecated storage 2025-08-06 00:02:50 +08:00
yangdx
c22315ea6d refactor: remove selective LLM cache clearing functionality 
- Remove optional 'modes' parameter from aclear_cache() and clear_cache() methods
- Replace deprecated drop_cache_by_modes() with drop() method for complete cache clearing
- Update API endpoint to ignore mode-specific parameters and clear all cache
- Simplify frontend clearCache() function to send empty request body

This change ensures all LLM cache is cleared together.
 2025-08-05 23:51:51 +08:00
yangdx
cc1f7118e7 Remove deprecated cache_by_modes functionality from all storage 2025-08-05 23:20:26 +08:00
yangdx
8294d6d1b7 Remove deprecated mode field from LLM cache schema 
- Drop mode column from LLM cache table
- Update primary key to exclude mode
- Remove mode from all SQL queries
- Deprecate mode-related methods
- Update schema migration logic
 2025-08-05 23:18:54 +08:00
yangdx
0b5c708660 Update storage implementation documentation 
- Add detailed storage type descriptions
- Remove Chroma from vector storage options
- Include recommended PostgreSQL version
- Add Memgraph to graph storage options
- Update performance comparison notes
 2025-08-05 18:03:51 +08:00
yangdx
0463963520 fix: include all query parameters in LLM cache hash key generation 
- Add missing query parameters (top_k, enable_rerank, max_tokens, etc.) to cache key generation in kg_query, naive_query, and extract_keywords_only functions
- Add queryparam field to CacheData structure and PostgreSQL storage for debugging
- Update PostgreSQL schema with automatic migration for queryparam JSONB column
- Prevent incorrect cache hits between queries with different parameters

Fixes issue where different query parameters incorrectly shared the same cached results.
 2025-08-05 18:03:10 +08:00
yangdx
cb75e6631e Remove quantized embedding info from LLM cache 
- Delete quantize_embedding function
- Delete dequantize_embedding function
- Remove embedding fields from CacheData
- Update save_to_cache to exclude embedding data
- Clean up unused quantization-related code
 2025-08-05 17:58:34 +08:00
Daniel.y
c7d17f13c1
Merge pull request #1914 from danielaskdd/feat-tiktoken-cache 
feat: add tiktoken cache directory support for offline deployment
 2025-08-05 14:25:10 +08:00
yangdx
8dd3069005 feat: add tiktoken cache directory support for offline deployment 
- Mount tiktoken cache volume in docker-compose
- Add TIKTOKEN_CACHE_DIR environment variable
- Update env.example with tiktoken cache configuration
- Added /data to gitignore
- Added tiktoken env example
 2025-08-05 14:22:19 +08:00
yangdx
01bce8c26e feat: add warning logs for deleting non-completed documents 2025-08-05 12:21:08 +08:00
Daniel.y
793e82ae89
Merge pull request #1913 from danielaskdd/fix-base64-for-embedding 
Feat: Change embedding formats from float to base64 for efficiency
 2025-08-05 11:48:53 +08:00
yangdx
6ff25210ea feat: improve Jina API error handling to show clean messages instead of HTML 2025-08-05 11:46:02 +08:00
yangdx
c5babf61d7 Feat: Change embedding formats from float to base64 for efficiency 
- Add base64 support for Jina embeddings
- Add base64 support for OpenAI embeddings
- Update env.example with new embedding options
 2025-08-05 11:38:40 +08:00
yangdx
4d492abf41 feat: implement temperature priority cascade for LLM bindings 
- Add global --temperature command line argument with env fallback
- Implement temperature priority for Ollama LLM binding:
  1. --ollama-llm-temperature (highest)
  2. OLLAMA_LLM_TEMPERATURE env var
  3. --temperature command arg
  4. TEMPERATURE env var (lowest)
- Implement same priority logic for OpenAI/Azure OpenAI LLM binding
- Ensure command line args always override environment variables
- Maintain backward compatibility with existing configurations
 2025-08-05 04:53:55 +08:00
Daniel.y
51deee3d82
Merge pull request #1910 from danielaskdd/openai-option 
feat: Add OpenAI LLM Options Support
 2025-08-05 03:55:23 +08:00
yangdx
adf7ec8e35 feat: Add OpenAI LLM Options support with BindingOptions framework 
- Add OpenAILLMOptions dataclass with full OpenAI API parameter support
- Integrate OpenAI options in config.py for automatic binding detection
- Update server functions to inject OpenAI options for openai/azure_openai bindings
 2025-08-05 03:47:26 +08:00
yangdx
3099748668 Add temperature fallback for Ollama LLM binding 
- Implement OLLAMA_LLM_TEMPERATURE env var
- Fallback to global TEMPERATURE if unset
- Remove redundant OllamaLLMOptions logic
- Update env.example with new setting
 2025-08-05 01:50:09 +08:00
Daniel.y
936809a084
Merge pull request #1909 from danielaskdd/fix-ollama 
Fix ollama stop option handling and enhance temperature configuration
 2025-08-04 23:51:29 +08:00
yangdx
e5e3f0f878 Fix(Ollama option): change stop option from string to list and add fallback global temperature setting 2025-08-04 19:43:14 +08:00
yangdx
f8a880ac66 Improved binding options testing and documentation 2025-08-04 18:21:55 +08:00
yangdx
7b3a9c09ca Fix: add missing colume to LLM cache of PostgreSQL implementation 2025-08-04 11:12:59 +08:00
yangdx
223f4cdf62 Mark deprecated fields with TODO comments 2025-08-04 11:11:57 +08:00
yangdx
3bac19df5a Bump core version to 1.4.7 and api version to 0198 2025-08-04 10:55:41 +08:00
yangdx
63496698a1 Fix: ensure data migration is handled by single-process 
- Wrap migration logic with get_data_init_lock() to ensure single-process execution
- Prevent race conditions when multiple processes start simultaneously
 2025-08-04 01:47:20 +08:00
yangdx
e04d8ed8a7 Improved storage drop logging with namespace details 
- Added namespace and workspace to drop logs
 2025-08-04 00:56:39 +08:00
Daniel.y
7de0276a57
Merge pull request #1904 from HKUDS/optimize-doc-delete 
feat(performance): Optimize Document Deletion with Entity/Relation Indexing
 2025-08-04 00:39:02 +08:00
yangdx
5513155808 Fix namespace tablename translate error 
- Reorder namespace table map for PostgreSQL
- Ensure specific namespaces come first
 2025-08-04 00:21:20 +08:00
yangdx
daf2633dc2 Bump api version to 0198 2025-08-03 23:04:42 +08:00
yangdx
2f1d1b69e5 Bump api version to 0197 2025-08-03 23:03:48 +08:00
yangdx
7505195303 fix: add full_entities and full_relations to clear_documents storage list 2025-08-03 23:02:58 +08:00
yangdx
952d1feb07 feat: Add support for KV_STORE_FULL_ENTITIES and KV_STORE_FULL_RELATIONS namespaces in PGKVStorage 
- Add LIGHTRAG_FULL_ENTITIES and LIGHTRAG_FULL_RELATIONS table schemas
- Implement complete CRUD operations for both namespaces
- Add automatic table creation and migration support
- Add SQL templates and namespace mappings
- Ensure workspace isolation and proper indexing
 2025-08-03 22:54:56 +08:00
yangdx
bf9a6d699b Fix(lightrag): Handle undirected edges in data migration 
The `_migrate_entity_relation_data` function previously processed directed edges from `get_all_edges`, which could lead to duplicates (e.g., (A,B) and (B,A)) and an incorrect relation count.

This commit normalizes edges by sorting their source and target nodes before adding them to the relation set. This ensures all edges are treated as undirected and are properly deduplicated.
 2025-08-03 22:14:24 +08:00