luxiang
fb4166ba2a
chore: compatible wit qdrant v1.7.3
2025-09-10 20:07:49 +08:00
Daniel.y
f064b950fc
Merge pull request #2027 from Matt23-star/main
...
Refactor: PostgreSQL
2025-09-09 15:12:35 +08:00
yangdx
f19cce16be
Fix incorrect variable name in NetworkXStorage file path
...
- Fix working_dir -> workspace_dir typo
- Correct GraphML file path generation
2025-09-04 18:31:53 +08:00
Matt23-star
24cb11f3f5
style: ruff-format
2025-08-29 21:09:14 -07:00
Hao Feng
b860ffe510
Merge branch 'main' into main
2025-08-29 21:03:37 -07:00
yangdx
f3989548b9
Fix MongoDB vector query embedding format compatibility
...
* Convert numpy arrays to lists
* Ensure MongoDB compatibility
2025-08-29 18:51:53 +08:00
yangdx
03d0fa3014
perf: add optional query_embedding parameter to avoid redundant embedding calls
2025-08-29 18:15:45 +08:00
yangdx
a923d378dd
Remove deprecated ID-based filtering from vector storage queries
...
- Remove ids param from QueryParam
- Simplify BaseVectorStorage.query signature
- Update all vector storage implementations
- Streamline PostgreSQL query templates
- Remove ID filtering from operate.py calls
2025-08-29 17:06:48 +08:00
Matt23-star
aa1ef3f053
feat: optimize database query methods for improved performance and readability
2025-08-28 16:18:15 -07:00
Matt23-star
9804a1885b
feat: refactor parameter handling in database queries to use lists for improved consistency
2025-08-28 16:17:35 -07:00
LinkinPony
ff4c747a2a
fix mismatch of 'error' and 'error_msg' in MongoDB
2025-08-26 10:43:56 +08:00
yangdx
059003c906
Rename allow_create to first_initialization for clarity
2025-08-23 02:34:39 +08:00
Albert Gil López
3fca3be09b
fix: Fix server startup issue with PipelineNotInitializedError
...
- Add allow_create parameter to get_namespace_data() to permit internal initialization
- initialize_pipeline_status() now uses allow_create=True to create the namespace
- External calls still get the error if pipeline_status is not initialized
- This maintains the improved error messages while allowing proper server startup
Fixes server startup failure reported in PR #1978
2025-08-22 10:55:56 +00:00
Albert Gil López
c66fc3483a
fix: Implement PipelineNotInitializedError usage in get_namespace_data
...
- Add PipelineNotInitializedError import to shared_storage.py
- Raise PipelineNotInitializedError when accessing uninitialized pipeline_status namespace
- This provides clear error messages to users about initialization requirements
- Other namespaces continue to be created dynamically as before
Addresses review feedback from PR #1978 about unused exception class
2025-08-22 02:52:51 +00:00
Albert Gil López
3a64b267cb
Merge upstream/main and resolve conflicts
2025-08-21 16:56:11 +00:00
Matt23-star
015e9ae3dd
Merge branch 'main' into feature/optimization
...
# Conflicts:
# lightrag/kg/postgres_impl.py
2025-08-20 16:05:38 +08:00
Matt23-star
874ddda605
feat: remove unused parameter from query methods across multiple implementations
2025-08-20 15:59:05 +08:00
yangdx
a4c4b1182a
Fix logging level usage in Redis retry decorator
...
* Replace string with logging.WARNING constant
2025-08-20 05:21:15 +08:00
Albert Gil López
f35963c020
feat: Add clear error messages for uninitialized storage
...
- Add StorageNotInitializedError and PipelineNotInitializedError exceptions
- Update JsonDocStatusStorage to raise clear errors when not initialized
- Update JsonKVStorage to raise clear errors when not initialized
- Error messages now include complete initialization instructions
- Helps users understand and fix initialization issues quickly
Addresses feedback from issue #1933 about improving error clarity
2025-08-19 06:41:52 +00:00
yangdx
cdfbd2114f
Merge branch 'main' into pg-optimization
2025-08-18 22:24:37 +08:00
yangdx
a9d6807432
Fix query windows size limitation for Milvus data migration
2025-08-18 16:29:03 +08:00
yangdx
47b8caaf64
Stop execution on validation errors in Milvus storage
...
• Stop execution on validation errors to prevent potential data loss
2025-08-18 14:15:07 +08:00
yangdx
dcec511f72
feat: increase file path length limit to 32768 and add schema migration for Milvus DB
...
- Bump path limit to 32768 chars
- Add migration detection logic
- Implement dual-client migration
- Auto-migrate old collections
2025-08-18 04:37:12 +08:00
Matt23-star
60564cf453
fix: correct parameter usage in database query for improved reliability
2025-08-17 13:50:41 +08:00
yangdx
d84715bae7
Improve MongoDB index migration with better conflict detection
...
• Enhanced conflict detection logic
• Improved index comparison method
2025-08-17 12:53:05 +08:00
yangdx
61469c0a56
Add Chinese pinyin sorting support across document operations
...
• Replace pyuca with centralized utils function
• Add pinyin sort keys for file paths
• Update MongoDB indexes with zh collation
• Migrate existing indexes for compatibility
• Support Chinese chars in Redis/JSON storage
• Keep PostgreSQL sorting order controled by Database Collate order
2025-08-17 12:45:48 +08:00
yangdx
185b576101
Fix parameter reference and apply code formatting improvements
2025-08-17 04:02:43 +08:00
Matt23-star
a0593ec1c9
feat: enhance query performance by restructuring relationships, entities, and chunks retrieval in PostgreSQL.
...
Fixed: duplicate items query
2025-08-16 22:49:54 +08:00
Matt23-star
6a7e3092ea
feat: optimize node and edge queries in PostgreSQL. query tables Directly
2025-08-16 22:37:48 +08:00
Matt23-star
a7da48e05c
feat: add batch size parameter to node and edge retrieval methods
2025-08-16 22:35:22 +08:00
yangdx
2a781dfb91
Update Neo4j database naming in env.example
2025-08-15 19:14:38 +08:00
yangdx
3a227e37b8
Add get_vectors_by_ids method to MongoVectorDBStorage
2025-08-15 16:53:14 +08:00
yangdx
7a7385a200
Add efficient vector retrieval by IDs to PGVectorStorage
2025-08-15 16:51:41 +08:00
yangdx
8f7031b882
Add get_vectors_by_ids method to QdrantVectorDBStorage
2025-08-15 16:46:52 +08:00
yangdx
a71499a180
Add get_vectors_by_ids method to MilvusVectorDBStorage
2025-08-15 16:36:50 +08:00
yangdx
1e2d5252d7
Add get_vectors_by_ids method and filter out vector data from query results
2025-08-15 16:32:26 +08:00
yangdx
f85e2aa4bf
Merge branch 'main' into pick-trunk-by-vector
2025-08-14 03:54:26 +08:00
yangdx
0b22ffb252
Refac: uniformly protected with the get_data_init_lock for all storage initializations
2025-08-14 03:46:19 +08:00
yangdx
f1dafa0d01
feat: KG related chunks selection by vector similarity
...
- Add env switch to toggle weighted polling vs vector-similarity strategy
- Implement similarity-based sorting with fallback to weighted
- Introduce batch vector read API for vector storage
- Implement vector store and retrive funtion for Nanovector DB
- Preserve default behavior (weighted polling selection method)
2025-08-13 18:16:42 +08:00
yangdx
578bdaa410
Pin pymilvus version to 2.5.2 to avoid Protobuf version warning
2025-08-12 18:22:00 +08:00
yangdx
5d1bc8b49d
Relocate client creation to the initialize method to prevent race conditions in multi-process mode.
2025-08-12 18:20:56 +08:00
yangdx
74783d7781
Remove redundant debug logging for Qdrant operations
2025-08-12 17:29:05 +08:00
yangdx
41f8ef05b9
Restore thread safety to MongoDB client manager
...
- Protected client creation with lock
- Protected client release with lock
2025-08-12 16:42:53 +08:00
yangdx
0b2c3d06c7
- Remove redundant collection listing check
2025-08-12 15:24:06 +08:00
yangdx
fc8ca1a706
Fix: add muti-process lock for initialize and drop method for all storage
2025-08-12 04:25:09 +08:00
yangdx
ca00b9c8ee
Fix: Resolve workspace isolation problem for PostgreSQL with multiple LightRAG instances
2025-08-12 01:27:05 +08:00
yangdx
d9c1f935f5
Fix: Resolve workspace isolation issues in in-memory database with multiple LightRAG instances
2025-08-12 01:26:09 +08:00
yangdx
095e0cbfa2
Refac: Add workspace infomation to all logger output for all storage type
2025-08-12 01:19:09 +08:00
yangdx
16c9a81f4c
feat: support config.ini for PostgreSQL vector index settings
...
- Add support for reading vector_index_type, hnsw_m, hnsw_ef, and ivfflat_lists from config.ini
- Maintain backward compatibility with environment variables
- Update config.ini.example with new PostgreSQL vector index options
- Follow existing configuration priority: env vars > config.ini > defaults
2025-08-08 02:55:49 +08:00
yangdx
dec4148075
Merge branch 'main' into Matt23-star/main
2025-08-08 02:24:34 +08:00
