gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3988 commits 5 branches 72 tags 705 MiB
Commit graph

637 commits

Author SHA1 Message Date
yangdx
4c2b4b4b6b Revert "Fix LLM cache handling for Redis to address document deletion scenarios." 
This reverts commit 14cda93988.
 2025-06-29 22:35:40 +08:00
yangdx
10cd9c90e7 Revert "Fix linting" 
This reverts commit abd9de2a63.
 2025-06-29 22:35:26 +08:00
yangdx
abd9de2a63 Fix linting 2025-06-29 15:15:49 +08:00
yangdx
14cda93988 Fix LLM cache handling for Redis to address document deletion scenarios. 
- Implements bulk scan for "extract" cache entries
- Maintains backward compatibility for normal IDs
 2025-06-29 15:13:42 +08:00
yangdx
e2824b721e Fix LLM cache handling for MongoKVStorage to address document deletion scenarios. 
- Support fetching all "default_" prefixed documents
- Maintain original behavior for other IDs
- Return dictionary of documents for "default"
- Keep backward compatibility
 2025-06-29 15:03:57 +08:00
yangdx
37bf341a69 Fix LLM cache handling for PGKVStorage to address document deletion scenarios. 
- Add dynamic cache_type field
- Support mode parameter for LLM cache
- Maintain backward compatibility
 2025-06-29 14:39:50 +08:00
yangdx
6d5e73a251 Merge branch 'kenspirit/main' 2025-06-29 00:30:58 +08:00
yangdx
28aedd8b3c Update comments 2025-06-29 00:30:39 +08:00
Ken Chen
4a953d6829 As Graph edges should be treated undirectional, fix incorrect upsert_edge method in MongoDBGraph 2025-06-28 21:03:54 +08:00
Ken Chen
5116d61eaa As Graph edges should be treated undirectional, fix incorrect has_edge method in MongoDBGraph 2025-06-28 20:48:30 +08:00
Ken Chen
73cc86662a Add two BFS subgraph search support for MongoDBGraph 2025-06-28 20:00:13 +08:00
Ken Chen
5739f52d29 Rewrite get_knowledge_graph with label * by degree 2025-06-28 17:10:39 +08:00
Ken Chen
d0f4eee404 Fix accidentally hardcoded edge collection name in searching upstream nodes 2025-06-28 16:25:44 +08:00
yangdx
b7f8c20e61 fix(postgres): use correct table for vector queries 
Change SQL templates from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
to fix "content_vector does not exist" error in vector operations.
 2025-06-28 15:36:54 +08:00
yangdx
2c47367975 Fix linting 2025-06-28 14:37:55 +08:00
yangdx
95c7a7d038 feat(db): Add data migration from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS 2025-06-28 14:37:47 +08:00
yangdx
3a8a99b73d feat(postgres): Implement text_chunks upsert for PGKVStorage 2025-06-28 14:37:35 +08:00
Ken Chen
6574dfb7ea Fix accidentally hardcode max depth in searching upstream nodes 2025-06-28 11:40:39 +08:00
Ken Chen
b586bdc02f Fix accidentally hardcode label in searching upstream nodes 2025-06-28 10:50:56 +08:00
Ken Chen
7c8f65d020 Add search on neighbor nodes which are source to selected one 2025-06-28 08:50:32 +08:00
yangdx
44daf51501 Fix edge_degree to handle non-existent nodes 2025-06-27 01:37:10 +08:00
Ken Chen
f40bc43d5e Fix nodes & edges are missing when retrieving knowledge subgraph by selecting particular node_id 2025-06-26 23:11:31 +08:00
yangdx
6e1ca8f461 Merge branch 'patch-1' 2025-06-26 18:45:23 +08:00
yangdx
687ccd4923 fix: optimize MongoDB aggregation pipeline to prevent memory limit errors 
- Move $limit operation early in pipeline for "*" queries to reduce memory usage
- Remove memory-intensive $sort operation for large dataset queries
- Add fallback mechanism for memory limit errors with simple query
- Implement additional safety checks to enforce max_nodes limit
- Improve error handling and logging for memory-related issues
 2025-06-26 14:37:04 +08:00
yangdx
d8b544ab6f Fix linting 2025-06-26 14:15:11 +08:00
yangdx
c51079335e Optimize node label retrieval with aggregation 
- Enable allowDiskUse for large datasets
 2025-06-26 14:14:52 +08:00
yangdx
d60db573dc Add allowDiskUse flag to MongoDB aggregations 
- Enable disk use for large aggregations
- Fix cursor handling for list_search_indexes
- Improve query performance for big datasets
- Update vector search index check
- Set proper length for to_list results
 2025-06-26 13:51:53 +08:00
yangdx
71565f4794 Add get_all method to MongoKVStorage 2025-06-26 13:51:15 +08:00
yangdx
d512db26e4 Fix MongoDB set handling in delete operations 2025-06-26 13:50:19 +08:00
yangdx
6364d076aa Enable MongoGraphStorage 2025-06-26 13:47:50 +08:00
Ken Chen
a3865caaea Implement get_nodes_by_chunk_ids and get_edges_by_chunk_ids, 2025-06-25 22:17:17 +08:00
Ken Chen
81cff6e97f
Merge branch 'HKUDS:main' into main 2025-06-25 20:35:55 +08:00
yangdx
72384f87c4 Remove deprecated code from Postgres_impl.py 
- Stop filtering out 'base' node labels
- Match any edge type in query to improve performance
 2025-06-25 12:53:07 +08:00
yangdx
109c2b48be Fix linting 2025-06-25 12:39:43 +08:00
yangdx
da46b341dc feat: Optimize document deletion performance 
- To enhance performance during document deletion, new batch-get methods, `get_nodes_by_chunk_ids` and `get_edges_by_chunk_ids`, have been added to the graph storage layer (`BaseGraphStorage` and its implementations). The [`adelete_by_doc_id`](lightrag/lightrag.py:1681) function now leverages these methods to avoid unnecessary iteration over the entire knowledge graph, significantly improving efficiency.
- Graph storage updated: Networkx, Neo4j, Postgres AGE
 2025-06-25 12:37:57 +08:00
Ken Chen
12054fa8d9
Merge branch 'HKUDS:main' into main 2025-06-24 20:20:54 +08:00
yangdx
fd69c895cd Remove debug looging from Neo4j 2025-06-24 01:03:02 +08:00
yangdx
64d0df0caa Remove debug logging from Neo4J operations 
- Delete node upsert debug log
- Remove edge upsert debug messages
 2025-06-24 00:45:17 +08:00
yangdx
cc12460b38 Fix: Silence PostgreSQL logs during idempotent graph initialization 2025-06-23 23:08:56 +08:00
zrguo
4937de8809 Update 2025-06-22 15:12:09 +08:00
Ken Chen
a047d966ab MongoGraph: Separate edges from node collection 2025-06-21 21:05:04 +08:00
Ken Chen
cf441aa84c Add missing methods for MongoGraphStorage 2025-06-15 21:22:32 +08:00
zrguo
ead82a8dbd update delete_by_doc_id 2025-06-09 18:52:34 +08:00
pablo-statsig
4a8202138c
Remove graspologic pip installation that no longer seems to be used 
As far as I can tell this is no longer actually used and its usage was removed in this commit:
83353ab9a6 (diff-a346bcfb05aab0cc0c0baa6018976f4efab339e8cade9f6f8fb658fcbd54ae2e)

Our systems are flagging this package as having a dependency on a package with a less permissive license so I would appreciate if it can be removed if its no longer needed.  Let me know if that is not the case.
 2025-06-05 18:01:00 -07:00
yangdx
ada2443653 Optimize default setting of PostgreSQL 2025-05-22 17:09:26 +08:00
yangdx
2ee809cf58 Increase PG connection pool to 20 2025-05-22 16:37:18 +08:00
zrguo
cf4bb148fb fix linting 2025-05-12 16:28:36 +08:00
zrguo
61a21f8d5d
Merge pull request #1325 from venkateshpabbati/main 
security fix
 2025-05-12 16:25:11 +08:00
yangdx
ebdc7cea49 Merge branch 'allow_max_connection_config' into pg-max-connection 2025-05-09 14:16:53 +08:00
Arjun Rao
6ebd76d5da bugfix: convert config val to int 2025-05-09 04:22:46 +10:00