yangdx
a0710e9d65
Merge branch 'main' into fix-extract-entity-concurrent-problem
2025-02-01 14:45:42 +08:00
yangdx
635d4fd9e4
Add lock to protect file write operations in NanoVectorDBStorage
...
- Introduce asyncio.Lock for save operations
- Ensure thread-safe file writes
2025-02-01 10:36:25 +08:00
yangdx
6a326e2783
Revert "Refactor embedding functions and add async query limit"
...
This reverts commit 21481dba8f
2025-02-01 10:36:25 +08:00
Gurjot Singh
2894e8faf2
Fix linting errors
2025-01-31 19:05:47 +05:30
Gurjot Singh
8a624e198a
Add faiss integration for storage
2025-01-31 19:00:36 +05:30
yangdx
389f4ee872
Shorten log message for cosine similarity threshold.
2025-01-31 15:33:41 +08:00
yangdx
21481dba8f
Refactor embedding functions and add async query limit
...
- Separate insert/query embedding funcs
- Add query-specific async limit
- Update storage classes to use new funcs
- Protect vector DB save with lock
- Improve config handling for thresholds
2025-01-31 15:00:56 +08:00
yangdx
06647438b2
Refactor threshold handling to use environment variables and global config settings for oracle, postgres and tidb
2025-01-29 23:47:57 +08:00
yangdx
20d6355a4a
Fix cosine threshold parameter setting error for chroma
2025-01-29 22:41:18 +08:00
yangdx
90c765c724
Fix linting
2025-01-29 22:14:18 +08:00
yangdx
5c7b2d7c9f
Merge from main
2025-01-29 22:13:01 +08:00
yangdx
c8b890547a
Add logging for query parameters in NanoVectorDBStorage.query
2025-01-29 21:36:31 +08:00
yangdx
7aedc08caf
Add RAG configuration options and enhance parameter configurability
...
- Add top-k and cosine-threshold parms for api server
- Update .env and cli parms handling with new parameters
- Improve splash screen display
- Update bash and storage classes to read new parameters from .env file.
2025-01-29 21:34:34 +08:00
yangdx
d0052456d4
Fix cosine threshold parameter setting error
2025-01-29 21:09:11 +08:00
ranfysvalle02
4c349c208d
+MDB KG
2025-01-29 07:31:34 -05:00
zrguo
80451af839
fix linting errors
2025-01-27 23:21:34 +08:00
Saifeddine ALOUI
56e9c9f4d5
Moved the storages to kg folder
2025-01-27 09:59:26 +01:00
Saifeddine ALOUI
52037205eb
Update tidb_impl.py
2025-01-27 09:40:50 +01:00
Saifeddine ALOUI
b2c1144219
Update redis_impl.py
2025-01-27 09:39:58 +01:00
Saifeddine ALOUI
b6068046ff
Update postgres_impl.py
2025-01-27 09:39:39 +01:00
Saifeddine ALOUI
57682389e2
Update postgres_impl_test.py
2025-01-27 09:38:32 +01:00
Saifeddine ALOUI
c7c56863b1
Update postgres_impl.py
2025-01-27 09:36:53 +01:00
Saifeddine ALOUI
ecadb71556
Update oracle_impl.py
2025-01-27 09:36:34 +01:00
Saifeddine ALOUI
3fdeeff8ba
Update neo4j_impl.py
2025-01-27 09:36:10 +01:00
Saifeddine ALOUI
7a5d058a57
Update mongo_impl.py
2025-01-27 09:35:50 +01:00
Saifeddine ALOUI
9390abb49b
Update milvus_impl.py
2025-01-27 09:35:26 +01:00
Saifeddine ALOUI
af245eb73e
Update age_impl.py
2025-01-27 09:35:08 +01:00
Saifeddine ALOUI
f307ed43f5
fixed linting
2025-01-27 02:10:24 +01:00
Saifeddine ALOUI
a68aebb124
translated docstrings to english and enhanced the webui
2025-01-27 02:07:06 +01:00
hyb
cd5b1dc98f
fix: light_server.py fix
2025-01-26 09:13:11 +08:00
hyb
3dba406644
feat: Added webui management, including file upload, text upload, Q&A query, graph database management (can view tags, view knowledge graph based on tags), system status (whether it is good, data storage status, model status, path),request /webui/index.html
2025-01-25 18:38:46 +08:00
Saifeddine ALOUI
06c9e4e454
Fixed missing imports bug and fixed linting
2025-01-25 00:55:07 +01:00
Saifeddine ALOUI
34018cb1e0
Separated llms from the main llm.py file and fixed some deprication bugs
2025-01-25 00:11:00 +01:00
Lukas Selch
c75ef82b66
Fixed workspace matching in oracle_impl.py
...
In the query for "node_degree", there was a typo that did not match the workspace of the edges, but only of the nodes.
2025-01-24 15:57:31 +01:00
hyb
e08905b398
feat: 增加redis KV存储，增加openai+neo4j+milvus+redis的demo测试，新增lightrag.py: RedisKVStorage，新增requirements.txt:aioredis依赖
2025-01-22 16:42:13 +08:00
zrguo
b84aab5cd0
Merge pull request #590 from jin38324/main
...
Enhance Robustness of insert Method with Pipeline Processing and Caching Mechanisms
2025-01-16 14:20:08 +08:00
jin
6ae8647285
support pipeline mode
2025-01-16 12:58:15 +08:00
jin
d5ae6669ea
support pipeline mode
2025-01-16 12:52:37 +08:00
jin
17a2ec2bc4
Merge branch 'HKUDS:main' into main
2025-01-16 09:59:27 +08:00
Samuel Chan
2b7d253117
Merge remote-tracking branch 'origin/main'
2025-01-15 12:09:05 +08:00
Samuel Chan
d91a330e9d
Enrich README.md for postgres usage, make some change to cater python version<12
2025-01-15 12:02:55 +08:00
zrguo
61349b0fe7
Merge pull request #581 from jiabin2wang20230918/fix_mongo
...
Fix for mongoDB document 16MB limit and add os env NEO4J_MAX_CONNECTION_POOL_SIZE for neo4j
2025-01-13 18:38:22 +08:00
bingo
1984da0fd6
add logger.debug for mongo_impl get_by_mode_and_id()
2025-01-13 07:27:30 +00:00
bingo
f28b90b2b3
1. add os env NEO4J_MAX_CONNECTION_POOL_SIZE to for neo4j ;
...
2. fix https://github.com/HKUDS/LightRAG/issues/580 issue for mongoDB document 16MB limit.
2025-01-13 07:06:01 +00:00
Samuel Chan
c016934021
Revise the AGE implementation on get_node_edges, to align with Neo4j behavior.
2025-01-12 21:38:39 +08:00
Samuel Chan
d03d6f5fc5
Revised the postgres implementation, to use attributes(node_id) rather than nodes to identify an entity. Which significantly reduced the table counts.
2025-01-11 09:30:19 +08:00
jin
85331e3fa2
update Oracle support
...
add cache support, fix bug
2025-01-10 11:36:28 +08:00
Samuel Chan
85d35b16d4
Merge branch 'HKUDS:main' into main
2025-01-06 12:53:06 +08:00
Samuel Chan
6ae27d8f06
Some enhancements:
...
- Enable the llm_cache storage to support get_by_mode_and_id, to improve the performance for using real KV server
- Provide an option for the developers to cache the LLM response when extracting entities for a document. Solving the paint point that sometimes the process failed, the processed chunks we need to call LLM again, money and time wasted. With the new option (by default not enabled) enabling, we can cache that result, can significantly save the time and money for beginners.
2025-01-06 12:50:05 +08:00
zrguo
6d7a01f29d
Merge pull request #542 from xiyihan0/main
...
添加选取Neo4j指定数据库功能的支持
2025-01-05 12:46:45 +08:00
