yangdx
da8655002a
Add composite indexes for workspace+id columns for PostgreSQL
2025-07-07 03:36:49 +08:00
yangdx
907f2313cd
Improve MongoDB vector index handling with workspace support
...
- Add workspace-specific index naming
- Store index name as instance variable
2025-07-07 03:19:41 +08:00
yangdx
033098c1bc
Feat: Add WORKSPACE support to all storage types
2025-07-07 00:57:21 +08:00
yangdx
86a00ed333
Remove deprecated storage type
2025-07-05 13:11:32 +08:00
yangdx
bdfd2d53c7
Fix linting
2025-07-05 11:43:45 +08:00
yangdx
2e2b9f3b48
Refactor setup.py to utilize pyproject.toml for project installation.
2025-07-05 11:19:00 +08:00
yangdx
99bb25a282
Update logger message
2025-07-05 08:21:54 +08:00
yangdx
2d78d74055
Fix milvus compatible check error
2025-07-05 07:55:36 +08:00
yangdx
b842a80aab
Fix linting
2025-07-05 07:09:01 +08:00
yangdx
9878dc7f51
fix: ensure Milvus collections are loaded before operations
...
- Resolves "collection not loaded" MilvusException errors
2025-07-05 00:40:25 +08:00
yangdx
7ec4ba5f9a
Fix: recover faiss_impl.py
2025-07-04 23:50:27 +08:00
yangdx
951ea3b851
Remvoe deprecated storage type
2025-07-04 21:48:18 +08:00
yangdx
6201c7fb56
Refactoring Milvus implementation
2025-07-04 21:42:10 +08:00
yangdx
810676d54c
Fix faisss import error
2025-07-04 19:12:23 +08:00
yangdx
531502677e
fix: Use create_time when update_time is 0 in PGKVStorage queries
2025-07-03 23:38:53 +08:00
yangdx
3355a0ce95
Fix create_time conflict in MongoKVStorage updates
2025-07-03 22:58:08 +08:00
yangdx
6c2ae40d7d
Refac: Enhance KG rebuild stability by incorporating create_time into the LLM cache
2025-07-03 17:08:29 +08:00
yangdx
70e154b0aa
Fix linting
2025-07-03 12:26:05 +08:00
yangdx
ff1b1c61c7
Implemented storage types: PostgreSQL and MongoDB
2025-07-03 11:46:24 +08:00
yangdx
e56734cb8b
Refac: Optimize document deletion performance
...
- Adding chunks_list to dock_status
- Adding llm_cache_list to text_chunks
- Implemented storage types: JsonKV and Redis
2025-07-03 04:18:25 +08:00
yangdx
86c9a0cda2
Fix linting
2025-07-02 16:29:43 +08:00
yangdx
271722405f
feat: Flatten LLM cache structure for improved recall efficiency
...
Refactored the LLM cache to a flat Key-Value (KV) structure, replacing the previous nested format. The old structure used the 'mode' as a key and stored specific cache content as JSON nested under it. This change significantly enhances cache recall efficiency.
2025-07-02 16:11:53 +08:00
yangdx
4c2b4b4b6b
Revert "Fix LLM cache handling for Redis to address document deletion scenarios."
...
This reverts commit 14cda93988
2025-06-29 22:35:40 +08:00
yangdx
10cd9c90e7
Revert "Fix linting"
...
This reverts commit abd9de2a63
2025-06-29 22:35:26 +08:00
yangdx
abd9de2a63
Fix linting
2025-06-29 15:15:49 +08:00
yangdx
14cda93988
Fix LLM cache handling for Redis to address document deletion scenarios.
...
- Implements bulk scan for "extract" cache entries
- Maintains backward compatibility for normal IDs
2025-06-29 15:13:42 +08:00
yangdx
e2824b721e
Fix LLM cache handling for MongoKVStorage to address document deletion scenarios.
...
- Support fetching all "default_" prefixed documents
- Maintain original behavior for other IDs
- Return dictionary of documents for "default"
- Keep backward compatibility
2025-06-29 15:03:57 +08:00
yangdx
37bf341a69
Fix LLM cache handling for PGKVStorage to address document deletion scenarios.
...
- Add dynamic cache_type field
- Support mode parameter for LLM cache
- Maintain backward compatibility
2025-06-29 14:39:50 +08:00
yangdx
6d5e73a251
Merge branch 'kenspirit/main'
2025-06-29 00:30:58 +08:00
yangdx
28aedd8b3c
Update comments
2025-06-29 00:30:39 +08:00
Ken Chen
4a953d6829
As Graph edges should be treated undirectional, fix incorrect upsert_edge method in MongoDBGraph
2025-06-28 21:03:54 +08:00
Ken Chen
5116d61eaa
As Graph edges should be treated undirectional, fix incorrect has_edge method in MongoDBGraph
2025-06-28 20:48:30 +08:00
Ken Chen
73cc86662a
Add two BFS subgraph search support for MongoDBGraph
2025-06-28 20:00:13 +08:00
Ken Chen
5739f52d29
Rewrite get_knowledge_graph with label * by degree
2025-06-28 17:10:39 +08:00
Ken Chen
d0f4eee404
Fix accidentally hardcoded edge collection name in searching upstream nodes
2025-06-28 16:25:44 +08:00
yangdx
b7f8c20e61
fix(postgres): use correct table for vector queries
...
Change SQL templates from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
to fix "content_vector does not exist" error in vector operations.
2025-06-28 15:36:54 +08:00
yangdx
2c47367975
Fix linting
2025-06-28 14:37:55 +08:00
yangdx
95c7a7d038
feat(db): Add data migration from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
2025-06-28 14:37:47 +08:00
yangdx
3a8a99b73d
feat(postgres): Implement text_chunks upsert for PGKVStorage
2025-06-28 14:37:35 +08:00
Ken Chen
6574dfb7ea
Fix accidentally hardcode max depth in searching upstream nodes
2025-06-28 11:40:39 +08:00
Ken Chen
b586bdc02f
Fix accidentally hardcode label in searching upstream nodes
2025-06-28 10:50:56 +08:00
Ken Chen
7c8f65d020
Add search on neighbor nodes which are source to selected one
2025-06-28 08:50:32 +08:00
yangdx
44daf51501
Fix edge_degree to handle non-existent nodes
2025-06-27 01:37:10 +08:00
Ken Chen
f40bc43d5e
Fix nodes & edges are missing when retrieving knowledge subgraph by selecting particular node_id
2025-06-26 23:11:31 +08:00
yangdx
6e1ca8f461
Merge branch 'patch-1'
2025-06-26 18:45:23 +08:00
yangdx
687ccd4923
fix: optimize MongoDB aggregation pipeline to prevent memory limit errors
...
- Move $limit operation early in pipeline for "*" queries to reduce memory usage
- Remove memory-intensive $sort operation for large dataset queries
- Add fallback mechanism for memory limit errors with simple query
- Implement additional safety checks to enforce max_nodes limit
- Improve error handling and logging for memory-related issues
2025-06-26 14:37:04 +08:00
yangdx
d8b544ab6f
Fix linting
2025-06-26 14:15:11 +08:00
yangdx
c51079335e
Optimize node label retrieval with aggregation
...
- Enable allowDiskUse for large datasets
2025-06-26 14:14:52 +08:00
yangdx
d60db573dc
Add allowDiskUse flag to MongoDB aggregations
...
- Enable disk use for large aggregations
- Fix cursor handling for list_search_indexes
- Improve query performance for big datasets
- Update vector search index check
- Set proper length for to_list results
2025-06-26 13:51:53 +08:00
yangdx
71565f4794
Add get_all method to MongoKVStorage
2025-06-26 13:51:15 +08:00
