Why this change is needed:
The finalize_share_data() function was not properly cleaning up workspace
lock-related global variables (_sync_locks, _workspace_async_locks, and
lock registry variables). This caused stale references to remain after
finalization, leading to EOFError or BrokenPipeError when trying to
re-initialize or when processes tried to use locks after the Manager
was shut down.
How it solves it:
1. Added comprehensive cleanup of all Manager.dict proxies before Manager
shutdown (_sync_locks, _lock_registry, _lock_registry_count,
_lock_cleanup_data)
2. Added cleanup of per-process _workspace_async_locks dictionary
3. Reset all lock-related globals to None at end of finalization:
- _workers, _lock_registry, _lock_registry_count, _lock_cleanup_data
- _registry_guard, _storage_keyed_lock, _sync_locks
- _workspace_async_locks, _earliest_mp_cleanup_time,
_last_mp_cleanup_time
Impact:
- Prevents EOFError/BrokenPipeError in production deployments
- Enables safe re-initialization after finalization
- Critical for proper resource cleanup in multi-process deployments
- Fixes memory leaks from stale lock references
Testing:
- Added 3 comprehensive tests in test_finalization_cleanup.py
- All 23 workspace lock tests pass (17 original + 3 bug fixes + 3 finalization)
- Tests verify clean re-initialization after finalization in both
single-process and multiprocess modes
Bug 1a - RuntimeError when _registry_guard is None:
- Added explicit check for _registry_guard initialization
- Now raises clear RuntimeError instead of cryptic TypeError
- Helps users understand they need to call initialize_share_data() first
Bug 1b - Workspace async_locks not visible across processes:
- Created new _workspace_async_locks dict for per-process storage
- Fixed issue where async_locks modifications in one process were invisible to others
- This is correct design since asyncio.Lock objects cannot be pickled/shared
Why per-process async_locks:
- asyncio.Lock objects cannot be shared across processes
- Each process needs its own asyncio.Lock instances for coroutine sync
- Cross-process sync is handled by Manager.RLock() in _sync_locks
- Within-process async sync is handled by per-process asyncio.Lock
Testing:
- All 17 existing workspace lock tests pass
- Added 3 new tests specifically for bug verification
- Total 20 tests passing
Impact:
- Fixes potential race conditions in multiprocess scenarios
- Ensures proper synchronization both across and within processes
- Maintains backward compatibility
