Commit graph

4556 commits

Author SHA1 Message Date
Daniel.y
ad7d7d0854
Merge pull request #1770 from danielaskdd/merge_lock_with_key
Refac: Optimize keyed lock cleanup logic with time and size tracking
2025-07-12 05:24:36 +08:00
yangdx
5ee509e671 Fix linting 2025-07-12 05:17:44 +08:00
yangdx
964293f21b Optimize lock cleanup with time tracking and intervals
- Add cleanup time tracking variables
- Implement minimum cleanup intervals
- Track earliest cleanup times
- Handle time rollback cases
- Improve cleanup logging
2025-07-12 04:34:26 +08:00
yangdx
39965d7ded Move merging stage back controled by max parallel insert semhore 2025-07-12 03:32:08 +08:00
yangdx
7490a18481 Optimize lock cleanup parameters 2025-07-12 03:10:03 +08:00
yangdx
3d8e6924bc Show lock clean up message 2025-07-12 02:58:05 +08:00
yangdx
22c36f2fd2 Optimize log messages 2025-07-12 02:41:31 +08:00
yangdx
a64c767298 optimize: improve lock cleanup performance with threshold-based strategy
- Add CLEANUP_THRESHOLD constant (100) to control cleanup frequency
- Modify _release_shared_raw_mp_lock to only scan when cleanup list exceeds threshold
- Modify _release_async_lock to only scan when cleanup list exceeds threshold
2025-07-11 23:43:40 +08:00
DavIvek
15c4bac87f wip fix Memgraph get_knowledge_graph issue by using mage function 2025-07-11 17:02:42 +02:00
yangdx
ad99d9ba5a Improve code organization and comments 2025-07-11 22:13:02 +08:00
yangdx
c52c451cf7 Fix linting 2025-07-11 20:40:50 +08:00
yangdx
3afdd1b67c Fix initial count error for multi-process lock with key 2025-07-11 20:39:08 +08:00
Marvin Schmidt
42a1da0041 fix(build): pyproject.toml setup 2025-07-11 12:01:34 +02:00
DavIvek
8d295bd294 wip fix Memgraph get_knowledge_graph issues 2025-07-11 11:55:53 +02:00
yangdx
c47747da9e Merge branch 'main' into merge_lock_with_key 2025-07-11 16:37:10 +08:00
yangdx
ef4870fda5 Combined entity and edge processing tasks and optimize merging with semaphore 2025-07-11 16:34:54 +08:00
DavIvek
63d554bad8 run pre-commit 2025-07-10 16:57:13 +02:00
DavIvek
a0c4d88b0d wip fix Memgraph get_knowledge_graph issues 2025-07-10 16:56:44 +02:00
al1k
0d69e40995
feat: add cloudflare workers ai integration for lightrag 2025-07-10 13:05:19 +06:00
al1k
f153330419
cleaned up files 2025-07-10 12:59:16 +06:00
al1k
6ef08e2302
feat: add cloudflare workers ai integration for lightrag 2025-07-10 12:58:35 +06:00
al1k
1aeac40c30
feat: add cloudflare workers ai integration for lightrag 2025-07-10 12:55:43 +06:00
al1k
c9830b38cb
remove log files during testing 2025-07-10 11:56:59 +06:00
minh nhan nguyen
dd830e9b9d
Merge branch 'HKUDS:main' into main 2025-07-10 11:25:51 +07:00
Daniel.y
ba0cffd853
Merge pull request #1753 from HKUDS/rerank
Add rerank optional for chunks
2025-07-09 16:06:55 +08:00
zrguo
b0479c078a fix process_chunks_unified() 2025-07-09 15:55:38 +08:00
yangdx
9aa2ed0837 Merge branch 'main' into rerank 2025-07-09 15:33:39 +08:00
yangdx
e457374224 Fix linting 2025-07-09 15:33:05 +08:00
yangdx
bfa0844ecb Update README 2025-07-09 15:17:05 +08:00
zrguo
e1541caea9 Update webui setting 2025-07-09 12:10:06 +08:00
yangdx
207f0a7f2a Merge branch 'main' into merge_lock_with_key 2025-07-09 09:25:28 +08:00
yangdx
cb3bfc0e5b Release semphore before merge stage 2025-07-09 09:24:44 +08:00
Daniel.y
feb30d8987
Merge pull request #1759 from antonvice/bugfix/handle-none-filepath
Fix: Handle NoneType error when processing documents without a file path
2025-07-09 09:21:04 +08:00
Anton Vice
b192f8c9a3 Fix: Handle NoneType error when processing documents without a file path
The document processing pipeline would crash with a TypeError when a document was submitted as raw text via the API, as the file_path attribute would be None. This change adds a check to handle the None case gracefully, preventing the crash and allowing text-based documents to be indexed correctly.
2025-07-08 19:35:22 -03:00
yangdx
4705a22861 Bump core version to 1.4.0 2025-07-09 04:43:20 +08:00
yangdx
2056c3c809 Increase default CHUNK_TOP_K from 5 to 15 2025-07-09 04:41:51 +08:00
yangdx
78033edabb Update env.example 2025-07-09 04:37:04 +08:00
yangdx
e9c3503f77 Update logger info 2025-07-09 04:36:52 +08:00
yangdx
5d4484882a Merge branch 'main' into rerank 2025-07-09 03:59:04 +08:00
Daniel.y
1497f334f6
Merge pull request #1703 from DavIvek/add-Memgraph-graph-db
Add Memgraph as a Supported Graph Database Backend.
2025-07-09 03:45:22 +08:00
Daniel.y
565b274490
Merge pull request #1758 from HKUDS/memgraph
Merge Memgraph branch
2025-07-09 03:43:25 +08:00
yangdx
14d51518dd Merge branch 'add-Memgraph-graph-db' into memgraph 2025-07-09 03:38:07 +08:00
yangdx
3a0249a6b9 Update env.example 2025-07-09 03:36:17 +08:00
DavIvek
1854d7c75a Merge remote-tracking branch 'upstream/memgraph' into add-Memgraph-graph-db 2025-07-08 21:00:24 +02:00
DavIvek
08eb68b8ed run pre-commit 2025-07-08 20:21:20 +02:00
yangdx
75ce636084 Merge branch 'main' into add-Memgraph-graph-db 2025-07-09 02:09:35 +08:00
DavIvek
4438897b6b add changes based on review 2025-07-08 16:28:06 +02:00
zrguo
d4651d59c1 Add rerank to server 2025-07-08 21:44:20 +08:00
minh nhan nguyen
0b6cc442c4
Create terminal_log.txt
logs of the terminal output
2025-07-08 18:02:49 +07:00
minh nhan nguyen
2156219b71
Create book.txt 2025-07-08 17:59:15 +07:00