yangdx
d54d0d55d9
Standardize empty workspace handling from "_" to "" across storage
...
* Unify empty workspace behavior by changing workspace from "_" to ""
* Fixed incorrect empty workspace detection in get_all_update_flags_status()
2025-11-17 12:54:33 +08:00
yangdx
fd486bc922
Refactor storage classes to use namespace instead of final_namespace
2025-11-17 12:54:33 +08:00
yangdx
926960e957
Refactor workspace handling to use default workspace and namespace locks
...
- Remove DB-specific workspace configs
- Add default workspace auto-setting
- Replace global locks with namespace locks
- Simplify pipeline status management
- Remove redundant graph DB locking
2025-11-17 12:54:33 +08:00
yangdx
411e92e6b9
Fix vector deletion logging to show actual deleted count
2025-10-27 14:22:16 +08:00
yangdx
9be22dd666
Preserve ordering in get_by_ids methods across all storage implementations
...
- Fix result ordering in vector stores
- Update KV storage get_by_ids methods
- Maintain order in doc status storage
- Return None for missing IDs
2025-10-11 12:37:59 +08:00
yangdx
03d0fa3014
perf: add optional query_embedding parameter to avoid redundant embedding calls
2025-08-29 18:15:45 +08:00
yangdx
a923d378dd
Remove deprecated ID-based filtering from vector storage queries
...
- Remove ids param from QueryParam
- Simplify BaseVectorStorage.query signature
- Update all vector storage implementations
- Streamline PostgreSQL query templates
- Remove ID filtering from operate.py calls
2025-08-29 17:06:48 +08:00
yangdx
f1dafa0d01
feat: KG related chunks selection by vector similarity
...
- Add env switch to toggle weighted polling vs vector-similarity strategy
- Implement similarity-based sorting with fallback to weighted
- Introduce batch vector read API for vector storage
- Implement vector store and retrive funtion for Nanovector DB
- Preserve default behavior (weighted polling selection method)
2025-08-13 18:16:42 +08:00
yangdx
d9c1f935f5
Fix: Resolve workspace isolation issues in in-memory database with multiple LightRAG instances
2025-08-12 01:26:09 +08:00
yangdx
44b7ce222e
feat: add default storage dependencies and optimize imports
...
- Add nano-vectordb and networkx to pyproject.toml dependencies
- Replace dynamic imports with direct imports for 4 default storage implementations
- Improve startup performance while maintaining backward compatibility
2025-07-24 16:14:26 +08:00
yangdx
033098c1bc
Feat: Add WORKSPACE support to all storage types
2025-07-07 00:57:21 +08:00
yangdx
045993f7d2
Remove deprecated search_by_prefix
2025-05-03 11:17:49 +08:00
yangdx
ca85f8a461
Convert timestamp to integer in vector DB storage
2025-05-01 23:25:47 +08:00
yangdx
8c7e9aceb2
Fix time handle bugs for nano vector db
2025-05-01 11:34:38 +08:00
yangdx
ca63386546
Increase embeding priority for query request
2025-04-28 20:10:39 +08:00
yangdx
ad087073aa
Optimize logger for storage
2025-04-10 01:07:06 +08:00
yangdx
95a8ee27ed
Fix linting
2025-03-31 23:22:27 +08:00
yangdx
1772e7a887
Add delete support to all storage implementation
2025-03-31 16:21:20 +08:00
yangdx
1df4b777d7
Add drop funtions to storage implementations
2025-03-30 15:17:57 +08:00
yangdx
15e060f854
Fix share storage update status handling problem of in memeory storage
2025-03-25 10:48:15 +08:00
yangdx
ff9cb2138d
Ensure thread safety in storage update callbacks
...
- Added storage lock in index_done_callback
- Fixed potential race conditions
2025-03-24 02:11:59 +08:00
ArindamRoy23
49dd5f936e
Merge branch 'HKUDS:main' into main
2025-03-11 20:53:00 +05:30
Roy
8aa9d0e6ca
Add optional ids filter to vector database query methods
...
- Updated query method signatures across multiple vector database implementations
- Added optional `ids` parameter to filter search results
- Consistent implementation across ChromaDB, Faiss, Milvus, MongoDB, NanoVectorDB, Oracle, Qdrant, and TiDB vector storage classes
2025-03-11 15:22:17 +00:00
zrguo
c26cb3a9ea
fix merge bugs
2025-03-11 16:05:04 +08:00
zrguo
e822f35c89
Fix edit entity and relation bugs
2025-03-07 14:39:06 +08:00
yangdx
e3a40c2fdb
Fix linting
2025-03-01 16:23:34 +08:00
yangdx
41eff2ca2f
Fix data persistence issue in NanoVectorDBStorage
2025-03-01 13:35:00 +08:00
yangdx
d4f6dcfd54
Improve multi-process data synchronization and persistence in storage implementations
...
• Remove _get_client() or _get_graph() from index_done_callback
• Add return value for index_done_callback
2025-03-01 12:41:30 +08:00
yangdx
d3de57c1e4
Add multi-process support for vector database and graph storage with lock flags
...
• Implement storage lock mechanism
• Add update flag handling
• Add cross-process reload detection
2025-03-01 10:37:05 +08:00
yangdx
fd76e00c6a
Refactor storage initialization to separate object creation from data loading
...
• Split __post_init__ and initialize()
• Move data loading to initialize()
• Add FastAPI lifespan integration
2025-03-01 03:48:19 +08:00
yangdx
cd7648791a
Fix linting
2025-02-28 01:25:59 +08:00
yangdx
291e0c1b14
revert vector and graph use local data(single process)
2025-02-28 01:14:25 +08:00
yangdx
64f22966a3
Fix linting
2025-02-27 19:05:51 +08:00
yangdx
1699b10a25
Refactor direct client/graph access to reduce redundant get calls in vector/graph ops
2025-02-27 15:14:54 +08:00
yangdx
f007ebf006
Refactor initialization logic for vector, KV and graph storage implementations
...
• Add try_initialize_namespace check
• Move init code out of storage locks
• Reduce redundant init conditions
• Simplify initialization flow
• Make init thread-safer
2025-02-27 14:55:07 +08:00
yangdx
7436c06f6c
Fix linting
2025-02-26 18:11:16 +08:00
yangdx
2c019dbc7b
Refactor storage initialization to avoid redundant intitial data loads across processes, show init logs to first load only
2025-02-26 12:28:49 +08:00
yangdx
2752a764ae
Refactor storage implementations to support both single and multi-process modes
...
• Add shared storage management module
• Support process/thread lock based on mode
2025-02-26 05:38:38 +08:00
yangdx
a642bb3190
refactor: use shared manager from main process for storage implementations.
2025-02-25 12:08:49 +08:00
yangdx
087d5770b0
feat(storage): Add shared memory support for file-based storage implementations
...
This commit adds multiprocessing shared memory support to file-based storage implementations:
- JsonDocStatusStorage
- JsonKVStorage
- NanoVectorDBStorage
- NetworkXStorage
Each storage module now uses module-level global variables with multiprocessing.Manager() to ensure data consistency across multiple uvicorn workers. All processes will see
updates immediately when data is modified through ainsert function.
2025-02-25 11:10:13 +08:00
Yannick Stephan
48a1ad9b3b
Merge pull request #883 from YanSte/fix-return-none
...
Optimised returns
2025-02-19 22:24:50 +01:00
Yannick Stephan
9277fe8c29
fixed return
2025-02-19 22:22:41 +01:00
Saifeddine ALOUI
45ee4dd08c
fixed linting
2025-02-19 20:50:39 +01:00
Saifeddine ALOUI
d3c443529c
Update nano_vector_db_impl.py
2025-02-19 19:49:41 +01:00
Yannick Stephan
2524e02428
remove tqdm and cleaned readme and ollama
2025-02-18 19:58:03 +01:00
Yannick Stephan
2b2c81a722
added some comments
2025-02-16 16:04:07 +01:00
Yannick Stephan
a1607bbcb9
Merge remote-tracking branch 'origin/main' into make-clear-what-implemented-or-not
...
# Conflicts:
# lightrag/base.py
# lightrag/kg/json_doc_status_impl.py
# lightrag/kg/mongo_impl.py
# lightrag/kg/postgres_impl.py
2025-02-16 15:29:16 +01:00
Yannick Stephan
0e7aff96bb
back to not making breaks
2025-02-16 15:08:50 +01:00
Yannick Stephan
a0844bca28
cleaned import
2025-02-16 14:45:45 +01:00
Yannick Stephan
3fef8201c6
added final, required methods and cleaned import
2025-02-16 14:38:09 +01:00
