gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5329 commits 5 branches 72 tags 705 MiB
Commit graph

847 commits

Author SHA1 Message Date
yangdx
baab992431 Update pymilvus dependency from 2.5.2 to >=2.6.2 2025-10-11 22:42:02 +08:00
yangdx
e1e4f1b02c Fix get_by_ids to return None for missing records consistently 2025-10-11 13:34:26 +08:00
yangdx
9be22dd666 Preserve ordering in get_by_ids methods across all storage implementations 
- Fix result ordering in vector stores
- Update KV storage get_by_ids methods
- Maintain order in doc status storage
- Return None for missing IDs
 2025-10-11 12:37:59 +08:00
yangdx
b3ed264707 Refactor PostgreSQL retry config to use centralized configuration 
• Move retry config to ClientManager
• Remove env var parsing from PostgreSQLDB
• Add config params to test setup
 2025-10-10 03:44:13 +08:00
yangdx
e758204ab2 Add PostgreSQL connection retry mechanism with comprehensive error handling 
• Implement connection retry with backoff
• Add transient error detection
• Pool management with timeout guards
 2025-10-10 03:06:01 +08:00
yangdx
f1e0110716 Merge branch 'kevinnkansah/main' 2025-10-07 23:04:59 +08:00
yangdx
f2c0b41e78 Make PostgreSQL statement_cache_size configuration optional 
• Remove forced int conversion
• Allow None values for cache size
• Add conditional parameter setting
 2025-10-07 22:57:21 +08:00
Aleks Vujić
dd8f44e621
Fixed typo in log message when creating new graph file 2025-10-07 14:30:05 +02:00
kevinnkansah
fdcb034da0 chore: distinguish settings 2025-10-06 12:01:40 +02:00
kevinnkansah
22a7b482c5 fix: renamed PostGreSQL options env variable and allowed LRU cache to be an optional env variable 2025-10-06 11:56:09 +02:00
kevinnkansah
d8a9617c0e fix: fix: asyncpg bouncer connection pool error 
Prepared statement caching is disabled by setting
`statement_cache_size=0` in the `asyncpg` connection pool parameters.
This is necessary to prevent
`asyncpg.exceptions.InvalidSQLStatementNameError` when using
transaction-level connection poolers like Supabase Supavisor or
pgbouncer, which do not support prepared statements.
 2025-10-06 00:36:25 +02:00
kevinnkansah
108cdbe133 feat: add options for PostGres connection 2025-10-05 23:29:04 +02:00
yangdx
457d51952e Add doc_name field to full docs storage 
- Store file_path in full_docs storage
- Update PostgreSQL implementation by map file_path to doc_name
- Other storage implementation automatically handles the new field
 2025-10-05 11:44:27 +08:00
yangdx
f99c4a3738 Fix graph truncation logic for depth-limited traversals 
• Only set truncated flag for node limit
• Keep depth limit info logging
• Improve log message clarity
• Fix false truncation detection
 2025-09-24 18:03:11 +08:00
yangdx
2adb8efdc7 Add duplicate document detection and skip processed files in scanning 
- Add get_doc_by_file_path to all storages
- Skip processed files in scan operation
- Check duplicates in upload endpoints
- Check duplicates in text insert APIs
- Return status info in duplicate responses
 2025-09-23 17:30:54 +08:00
yangdx
6b3a341977 Increase default PostgreSQL max connections from 20 to 50 2025-09-22 18:11:28 +08:00
yangdx
040b0c8620 Fix Neo4J index creation to check state instead of analyzer 
• Check index state not analyzer
• Skip if index is ONLINE
• Recreate if state not ONLINE
• Simplify recreation logic
 2025-09-20 23:51:50 +08:00
yangdx
5da1df3b19 Fix linting 2025-09-20 15:30:27 +08:00
yangdx
8e2a1fa59e Enhance Neo4j fulltext search with Chinese language support 
• Add CJK analyzer for Chinese text
• Auto-detect Chinese characters
• Recreate index if needed
• Separate Chinese/Latin search logic
• Improve fallback for Chinese queries
 2025-09-20 15:19:22 +08:00
yangdx
9330ccb14e Fix graph truncation logging to correctly identify truncation cause 2025-09-20 13:33:19 +08:00
yangdx
1dd164a122 Fix graph truncation detection for depth-limited BFS 
- Track unexplored neighbors at max depth
- Improve truncation flag accuracy
 2025-09-20 13:12:25 +08:00
yangdx
3296bcb553 Add high-performance label search methods to PostgreSQL graph storage 
- Add get_popular_labels() method
- Add search_labels() with fuzzy matching
- Use native SQL for better performance
- Include proper scoring and ranking
 2025-09-20 12:39:53 +08:00
yangdx
6f85bd6b19 Add workspace-aware MongoDB indexing and Atlas Search support 
• Add workspace attribute to storage classes
• Use workspace-specific index names
• Implement Atlas Search with fallbacks
• Add entity search and popular labels
• Improve index migration strategy
 2025-09-20 12:38:41 +08:00
yangdx
223397a247 Add label search and popularity methods to MemgraphStorage 
• Get popular labels by node degree
• Search labels with fuzzy matching
• Sort by relevance and connection count
 2025-09-20 12:38:04 +08:00
yangdx
e14cee69a3 Fix Neo4j typo and add fulltext search with performance optimizations 
- Fix NEO4J_DATABASE typo in env.example
- Add fulltext index for entity searches
- Implement get_popular_labels method
- Add search_labels with fuzzy matching
- Simplify B-Tree index creation logic
 2025-09-20 12:37:13 +08:00
yangdx
9db8f2fce5 feat: Add popular labels and search APIs with history management 
- Add popular/search label endpoints
- Implement SearchHistoryManager utility
- Replace client-side with server search
- Add graph data version tracking
- Update UI for better label discovery
 2025-09-20 02:03:47 +08:00
yangdx
43f6fcea6c Fix linting 2025-09-12 17:00:53 +08:00
luxiang
fb4166ba2a chore: compatible wit qdrant v1.7.3 2025-09-10 20:07:49 +08:00
Daniel.y
f064b950fc
Merge pull request #2027 from Matt23-star/main 
Refactor: PostgreSQL
 2025-09-09 15:12:35 +08:00
yangdx
f19cce16be Fix incorrect variable name in NetworkXStorage file path 
- Fix working_dir -> workspace_dir typo
- Correct GraphML file path generation
 2025-09-04 18:31:53 +08:00
Matt23-star
24cb11f3f5 style: ruff-format 2025-08-29 21:09:14 -07:00
Hao Feng
b860ffe510
Merge branch 'main' into main 2025-08-29 21:03:37 -07:00
yangdx
f3989548b9 Fix MongoDB vector query embedding format compatibility 
* Convert numpy arrays to lists
* Ensure MongoDB compatibility
 2025-08-29 18:51:53 +08:00
yangdx
03d0fa3014 perf: add optional query_embedding parameter to avoid redundant embedding calls 2025-08-29 18:15:45 +08:00
yangdx
a923d378dd Remove deprecated ID-based filtering from vector storage queries 
- Remove ids param from QueryParam
- Simplify BaseVectorStorage.query signature
- Update all vector storage implementations
- Streamline PostgreSQL query templates
- Remove ID filtering from operate.py calls
 2025-08-29 17:06:48 +08:00
Matt23-star
aa1ef3f053 feat: optimize database query methods for improved performance and readability 2025-08-28 16:18:15 -07:00
Matt23-star
9804a1885b feat: refactor parameter handling in database queries to use lists for improved consistency 2025-08-28 16:17:35 -07:00
LinkinPony
ff4c747a2a fix mismatch of 'error' and 'error_msg' in MongoDB 2025-08-26 10:43:56 +08:00
yangdx
059003c906 Rename allow_create to first_initialization for clarity 2025-08-23 02:34:39 +08:00
Albert Gil López
3fca3be09b fix: Fix server startup issue with PipelineNotInitializedError 
- Add allow_create parameter to get_namespace_data() to permit internal initialization
- initialize_pipeline_status() now uses allow_create=True to create the namespace
- External calls still get the error if pipeline_status is not initialized
- This maintains the improved error messages while allowing proper server startup

Fixes server startup failure reported in PR #1978
 2025-08-22 10:55:56 +00:00
Albert Gil López
c66fc3483a fix: Implement PipelineNotInitializedError usage in get_namespace_data 
- Add PipelineNotInitializedError import to shared_storage.py
- Raise PipelineNotInitializedError when accessing uninitialized pipeline_status namespace
- This provides clear error messages to users about initialization requirements
- Other namespaces continue to be created dynamically as before

Addresses review feedback from PR #1978 about unused exception class
 2025-08-22 02:52:51 +00:00
Albert Gil López
3a64b267cb Merge upstream/main and resolve conflicts 2025-08-21 16:56:11 +00:00
Matt23-star
015e9ae3dd Merge branch 'main' into feature/optimization 
# Conflicts:
#	lightrag/kg/postgres_impl.py
 2025-08-20 16:05:38 +08:00
Matt23-star
874ddda605 feat: remove unused parameter from query methods across multiple implementations 2025-08-20 15:59:05 +08:00
yangdx
a4c4b1182a Fix logging level usage in Redis retry decorator 
* Replace string with logging.WARNING constant
 2025-08-20 05:21:15 +08:00
Albert Gil López
f35963c020 feat: Add clear error messages for uninitialized storage 
- Add StorageNotInitializedError and PipelineNotInitializedError exceptions
- Update JsonDocStatusStorage to raise clear errors when not initialized
- Update JsonKVStorage to raise clear errors when not initialized
- Error messages now include complete initialization instructions
- Helps users understand and fix initialization issues quickly

Addresses feedback from issue #1933 about improving error clarity
 2025-08-19 06:41:52 +00:00
yangdx
cdfbd2114f Merge branch 'main' into pg-optimization 2025-08-18 22:24:37 +08:00
yangdx
a9d6807432 Fix query windows size limitation for Milvus data migration 2025-08-18 16:29:03 +08:00
yangdx
47b8caaf64 Stop execution on validation errors in Milvus storage 
• Stop execution on validation errors to prevent potential data loss
 2025-08-18 14:15:07 +08:00
yangdx
dcec511f72 feat: increase file path length limit to 32768 and add schema migration for Milvus DB 
- Bump path limit to 32768 chars
- Add migration detection logic
- Implement dual-client migration
- Auto-migrate old collections
 2025-08-18 04:37:12 +08:00