gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5096 commits 5 branches 72 tags 705 MiB
Commit graph

834 commits

Author SHA1 Message Date
yangdx
f99c4a3738 Fix graph truncation logic for depth-limited traversals 
• Only set truncated flag for node limit
• Keep depth limit info logging
• Improve log message clarity
• Fix false truncation detection
 2025-09-24 18:03:11 +08:00
yangdx
2adb8efdc7 Add duplicate document detection and skip processed files in scanning 
- Add get_doc_by_file_path to all storages
- Skip processed files in scan operation
- Check duplicates in upload endpoints
- Check duplicates in text insert APIs
- Return status info in duplicate responses
 2025-09-23 17:30:54 +08:00
yangdx
6b3a341977 Increase default PostgreSQL max connections from 20 to 50 2025-09-22 18:11:28 +08:00
yangdx
040b0c8620 Fix Neo4J index creation to check state instead of analyzer 
• Check index state not analyzer
• Skip if index is ONLINE
• Recreate if state not ONLINE
• Simplify recreation logic
 2025-09-20 23:51:50 +08:00
yangdx
5da1df3b19 Fix linting 2025-09-20 15:30:27 +08:00
yangdx
8e2a1fa59e Enhance Neo4j fulltext search with Chinese language support 
• Add CJK analyzer for Chinese text
• Auto-detect Chinese characters
• Recreate index if needed
• Separate Chinese/Latin search logic
• Improve fallback for Chinese queries
 2025-09-20 15:19:22 +08:00
yangdx
9330ccb14e Fix graph truncation logging to correctly identify truncation cause 2025-09-20 13:33:19 +08:00
yangdx
1dd164a122 Fix graph truncation detection for depth-limited BFS 
- Track unexplored neighbors at max depth
- Improve truncation flag accuracy
 2025-09-20 13:12:25 +08:00
yangdx
3296bcb553 Add high-performance label search methods to PostgreSQL graph storage 
- Add get_popular_labels() method
- Add search_labels() with fuzzy matching
- Use native SQL for better performance
- Include proper scoring and ranking
 2025-09-20 12:39:53 +08:00
yangdx
6f85bd6b19 Add workspace-aware MongoDB indexing and Atlas Search support 
• Add workspace attribute to storage classes
• Use workspace-specific index names
• Implement Atlas Search with fallbacks
• Add entity search and popular labels
• Improve index migration strategy
 2025-09-20 12:38:41 +08:00
yangdx
223397a247 Add label search and popularity methods to MemgraphStorage 
• Get popular labels by node degree
• Search labels with fuzzy matching
• Sort by relevance and connection count
 2025-09-20 12:38:04 +08:00
yangdx
e14cee69a3 Fix Neo4j typo and add fulltext search with performance optimizations 
- Fix NEO4J_DATABASE typo in env.example
- Add fulltext index for entity searches
- Implement get_popular_labels method
- Add search_labels with fuzzy matching
- Simplify B-Tree index creation logic
 2025-09-20 12:37:13 +08:00
yangdx
9db8f2fce5 feat: Add popular labels and search APIs with history management 
- Add popular/search label endpoints
- Implement SearchHistoryManager utility
- Replace client-side with server search
- Add graph data version tracking
- Update UI for better label discovery
 2025-09-20 02:03:47 +08:00
yangdx
43f6fcea6c Fix linting 2025-09-12 17:00:53 +08:00
luxiang
fb4166ba2a chore: compatible wit qdrant v1.7.3 2025-09-10 20:07:49 +08:00
Daniel.y
f064b950fc
Merge pull request #2027 from Matt23-star/main 
Refactor: PostgreSQL
 2025-09-09 15:12:35 +08:00
yangdx
f19cce16be Fix incorrect variable name in NetworkXStorage file path 
- Fix working_dir -> workspace_dir typo
- Correct GraphML file path generation
 2025-09-04 18:31:53 +08:00
Matt23-star
24cb11f3f5 style: ruff-format 2025-08-29 21:09:14 -07:00
Hao Feng
b860ffe510
Merge branch 'main' into main 2025-08-29 21:03:37 -07:00
yangdx
f3989548b9 Fix MongoDB vector query embedding format compatibility 
* Convert numpy arrays to lists
* Ensure MongoDB compatibility
 2025-08-29 18:51:53 +08:00
yangdx
03d0fa3014 perf: add optional query_embedding parameter to avoid redundant embedding calls 2025-08-29 18:15:45 +08:00
yangdx
a923d378dd Remove deprecated ID-based filtering from vector storage queries 
- Remove ids param from QueryParam
- Simplify BaseVectorStorage.query signature
- Update all vector storage implementations
- Streamline PostgreSQL query templates
- Remove ID filtering from operate.py calls
 2025-08-29 17:06:48 +08:00
Matt23-star
aa1ef3f053 feat: optimize database query methods for improved performance and readability 2025-08-28 16:18:15 -07:00
Matt23-star
9804a1885b feat: refactor parameter handling in database queries to use lists for improved consistency 2025-08-28 16:17:35 -07:00
LinkinPony
ff4c747a2a fix mismatch of 'error' and 'error_msg' in MongoDB 2025-08-26 10:43:56 +08:00
yangdx
059003c906 Rename allow_create to first_initialization for clarity 2025-08-23 02:34:39 +08:00
Albert Gil López
3fca3be09b fix: Fix server startup issue with PipelineNotInitializedError 
- Add allow_create parameter to get_namespace_data() to permit internal initialization
- initialize_pipeline_status() now uses allow_create=True to create the namespace
- External calls still get the error if pipeline_status is not initialized
- This maintains the improved error messages while allowing proper server startup

Fixes server startup failure reported in PR #1978
 2025-08-22 10:55:56 +00:00
Albert Gil López
c66fc3483a fix: Implement PipelineNotInitializedError usage in get_namespace_data 
- Add PipelineNotInitializedError import to shared_storage.py
- Raise PipelineNotInitializedError when accessing uninitialized pipeline_status namespace
- This provides clear error messages to users about initialization requirements
- Other namespaces continue to be created dynamically as before

Addresses review feedback from PR #1978 about unused exception class
 2025-08-22 02:52:51 +00:00
Albert Gil López
3a64b267cb Merge upstream/main and resolve conflicts 2025-08-21 16:56:11 +00:00
Matt23-star
015e9ae3dd Merge branch 'main' into feature/optimization 
# Conflicts:
#	lightrag/kg/postgres_impl.py
 2025-08-20 16:05:38 +08:00
Matt23-star
874ddda605 feat: remove unused parameter from query methods across multiple implementations 2025-08-20 15:59:05 +08:00
yangdx
a4c4b1182a Fix logging level usage in Redis retry decorator 
* Replace string with logging.WARNING constant
 2025-08-20 05:21:15 +08:00
Albert Gil López
f35963c020 feat: Add clear error messages for uninitialized storage 
- Add StorageNotInitializedError and PipelineNotInitializedError exceptions
- Update JsonDocStatusStorage to raise clear errors when not initialized
- Update JsonKVStorage to raise clear errors when not initialized
- Error messages now include complete initialization instructions
- Helps users understand and fix initialization issues quickly

Addresses feedback from issue #1933 about improving error clarity
 2025-08-19 06:41:52 +00:00
yangdx
cdfbd2114f Merge branch 'main' into pg-optimization 2025-08-18 22:24:37 +08:00
yangdx
a9d6807432 Fix query windows size limitation for Milvus data migration 2025-08-18 16:29:03 +08:00
yangdx
47b8caaf64 Stop execution on validation errors in Milvus storage 
• Stop execution on validation errors to prevent potential data loss
 2025-08-18 14:15:07 +08:00
yangdx
dcec511f72 feat: increase file path length limit to 32768 and add schema migration for Milvus DB 
- Bump path limit to 32768 chars
- Add migration detection logic
- Implement dual-client migration
- Auto-migrate old collections
 2025-08-18 04:37:12 +08:00
Matt23-star
60564cf453 fix: correct parameter usage in database query for improved reliability 2025-08-17 13:50:41 +08:00
yangdx
d84715bae7 Improve MongoDB index migration with better conflict detection 
• Enhanced conflict detection logic
• Improved index comparison method
 2025-08-17 12:53:05 +08:00
yangdx
61469c0a56 Add Chinese pinyin sorting support across document operations 
• Replace pyuca with centralized utils function
• Add pinyin sort keys for file paths
• Update MongoDB indexes with zh collation
• Migrate existing indexes for compatibility
• Support Chinese chars in Redis/JSON storage
• Keep PostgreSQL sorting order controled by Database Collate order
 2025-08-17 12:45:48 +08:00
yangdx
185b576101 Fix parameter reference and apply code formatting improvements 2025-08-17 04:02:43 +08:00
Matt23-star
a0593ec1c9 feat: enhance query performance by restructuring relationships, entities, and chunks retrieval in PostgreSQL. 
Fixed: duplicate items query
 2025-08-16 22:49:54 +08:00
Matt23-star
6a7e3092ea feat: optimize node and edge queries in PostgreSQL. query tables Directly 2025-08-16 22:37:48 +08:00
Matt23-star
a7da48e05c feat: add batch size parameter to node and edge retrieval methods 2025-08-16 22:35:22 +08:00
yangdx
2a781dfb91 Update Neo4j database naming in env.example 2025-08-15 19:14:38 +08:00
yangdx
3a227e37b8 Add get_vectors_by_ids method to MongoVectorDBStorage 2025-08-15 16:53:14 +08:00
yangdx
7a7385a200 Add efficient vector retrieval by IDs to PGVectorStorage 2025-08-15 16:51:41 +08:00
yangdx
8f7031b882 Add get_vectors_by_ids method to QdrantVectorDBStorage 2025-08-15 16:46:52 +08:00
yangdx
a71499a180 Add get_vectors_by_ids method to MilvusVectorDBStorage 2025-08-15 16:36:50 +08:00
yangdx
1e2d5252d7 Add get_vectors_by_ids method and filter out vector data from query results 2025-08-15 16:32:26 +08:00