LightRAG/lightrag
chengjie 0bd162a416 fix: ensure finalize_share_data properly cleans up workspace locks
Why this change is needed:
The finalize_share_data() function was not properly cleaning up workspace
lock-related global variables (_sync_locks, _workspace_async_locks, and
lock registry variables). This caused stale references to remain after
finalization, leading to EOFError or BrokenPipeError when trying to
re-initialize or when processes tried to use locks after the Manager
was shut down.

How it solves it:
1. Added comprehensive cleanup of all Manager.dict proxies before Manager
   shutdown (_sync_locks, _lock_registry, _lock_registry_count,
   _lock_cleanup_data)
2. Added cleanup of per-process _workspace_async_locks dictionary
3. Reset all lock-related globals to None at end of finalization:
   - _workers, _lock_registry, _lock_registry_count, _lock_cleanup_data
   - _registry_guard, _storage_keyed_lock, _sync_locks
   - _workspace_async_locks, _earliest_mp_cleanup_time,
     _last_mp_cleanup_time

Impact:
- Prevents EOFError/BrokenPipeError in production deployments
- Enables safe re-initialization after finalization
- Critical for proper resource cleanup in multi-process deployments
- Fixes memory leaks from stale lock references

Testing:
- Added 3 comprehensive tests in test_finalization_cleanup.py
- All 23 workspace lock tests pass (17 original + 3 bug fixes + 3 finalization)
- Tests verify clean re-initialization after finalization in both
  single-process and multiprocess modes
2025-11-11 00:23:42 +08:00
..
api Bump API version to 0253 2025-11-09 14:42:22 +08:00
evaluation Update LLM cache migration docs and improve UX prompts 2025-11-08 23:48:19 +08:00
kg fix: ensure finalize_share_data properly cleans up workspace locks 2025-11-11 00:23:42 +08:00
llm Refactor keyword_extraction from kwargs to explicit parameter 2025-11-09 12:02:17 +08:00
tools Add concurrency warning for JsonKVStorage in cleanup tool 2025-11-09 23:04:04 +08:00
__init__.py Bump core version to 1.4.9.9 and API to 0252 2025-11-08 11:27:26 +08:00
base.py Remove unused chunk-based node/edge retrieval methods 2025-11-06 18:17:10 +08:00
constants.py Refactor entity merging with unified attribute merge function 2025-10-27 00:04:17 +08:00
exceptions.py Add Qdrant legacy collection migration with workspace support 2025-10-30 19:16:33 +08:00
lightrag.py Fix: Remove redundant entity/relation chunk deletions 2025-11-07 02:56:16 +08:00
namespace.py Add entity/relation chunk tracking with configurable source ID limits 2025-10-20 15:24:15 +08:00
operate.py Prohibit direct access to internal functions of EmbeddingFunc. 2025-11-08 01:43:36 +08:00
prompt.py Fix typo in 'equipment' in prompt.py 2025-10-22 11:13:22 +08:00
rerank.py fix: Resolve default rerank config problem when env var missing 2025-08-23 01:07:59 +08:00
types.py
utils.py Add mandatory dimension parameter handling for Jina API compliance 2025-11-07 21:08:34 +08:00
utils_graph.py Improve entity merge logging by removing redundant message and fixing typo 2025-10-31 17:16:59 +08:00