gmakstutis/ragflow
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4547 commits 2 branches 38 tags 282 MiB
Commit graph

241 commits

Author SHA1 Message Date
“hu
1c8dcf7ac6 support dev setting to control if sync deleted 2025-11-26 08:35:31 +08:00
Stephen Hu
3f1bc60bbe Update sync_data_source.py 2025-11-24 12:59:12 +08:00
Stephen Hu
7b424c70fc fix when re index will delete the files not in remote 
1. add method in document service to query doc ids by src
2. add method in connector service to record delete
3. when re index support delete
 2025-11-24 09:11:37 +08:00
Levi
f0a14f5fce
Add Moodle data source integration (#11325) 
### What problem does this PR solve?

This PR adds a native Moodle connector to sync content (courses,
resources, forums, assignments, pages, books) into RAGFlow.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
 2025-11-21 19:58:49 +08:00
Yongteng Lei
13e212c856
Feat: add Jira connector (#11285) 
### What problem does this PR solve?

Add Jira connector.

<img width="978" height="925" alt="image"
src="https://github.com/user-attachments/assets/78bb5c77-2710-4569-a76e-9087ca23b227"
/>

---

<img width="1903" height="489" alt="image"
src="https://github.com/user-attachments/assets/193bc5c5-f751-4bd5-883a-2173282c2b96"
/>

---

<img width="1035" height="925" alt="image"
src="https://github.com/user-attachments/assets/1a0aec19-30eb-4ada-9283-61d1c915f59d"
/>

---

<img width="1905" height="601" alt="image"
src="https://github.com/user-attachments/assets/3dde1062-3f27-4717-8e09-fd5fd5e64171"
/>

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-17 09:38:04 +08:00
buua436
e8f1a245a6
Feat:update check_embedding api (#11254) 
### What problem does this PR solve?
pr: 
#10854
change:
update check_embedding api

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-13 18:48:25 +08:00
Yongteng Lei
908450509f
Feat: add fault-tolerant mechanism to RAPTOR (#11206) 
### What problem does this PR solve?

Add fault-tolerant mechanism to RAPTOR.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-13 18:48:07 +08:00
Jin Hai
296476ab89
Refactor function name (#11210) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-12 19:00:15 +08:00
Yongteng Lei
8ae562504b
Fix: GraphRAG and RAPTOR tasks do not affect document status (#11194) 
### What problem does this PR solve?

GraphRAG and RAPTOR tasks do not affect document status.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-12 12:03:41 +08:00
Kevin Hu
c30ffb5716
Fix: ollama model list issue. (#11175) 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-11 19:46:41 +08:00
Yongteng Lei
9213568692
Feat: add mechanism to check cancellation in Agent (#10766) 
### What problem does this PR solve?

Add mechanism to check cancellation in Agent.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-11 17:36:48 +08:00
Kevin Hu
f441f8ffc2
Fix: waitForResponse component. (#11172) 
### What problem does this PR solve?

#10056

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
 2025-11-11 16:58:47 +08:00
Yongteng Lei
df16a80f25
Feat: add initial Google Drive connector support (#11147) 
### What problem does this PR solve?

This feature is primarily ported from the
[Onyx](https://github.com/onyx-dot-app/onyx) project with necessary
modifications. Thanks for such a brilliant project.

Minor: consistently use `google_drive` rather than `google_driver`.

<img width="566" height="731" alt="image"
src="https://github.com/user-attachments/assets/6f64e70e-881e-42c7-b45f-809d3e0024a4"
/>

<img width="904" height="830" alt="image"
src="https://github.com/user-attachments/assets/dfa7d1ef-819a-4a82-8c52-0999f48ed4a6"
/>

<img width="911" height="869" alt="image"
src="https://github.com/user-attachments/assets/39e792fb-9fbe-4f3d-9b3c-b2265186bc22"
/>

<img width="947" height="323" alt="image"
src="https://github.com/user-attachments/assets/27d70e96-d9c0-42d9-8c89-276919b6d61d"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-10 19:15:02 +08:00
Kevin Hu
d207291217
Fix: add download stats to kb logs. (#11112) 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-10 13:28:07 +08:00
Lynn
d016a06fd5
Feat/monitor task (#11116) 
### What problem does this PR solve?

Show task executor.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-10 12:51:39 +08:00
Kevin Hu
dd1c8c5779
Feat: add auto parse to connector. (#11099) 
### What problem does this PR solve?

#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-07 16:49:29 +08:00
Kevin Hu
34283d4db4
Feat: add data source to pipleline logs . (#11075) 
### What problem does this PR solve?

#10953

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-07 11:43:59 +08:00
Jin Hai
af98763e27
Admin: add 'show version' (#11079) 
### What problem does this PR solve?

```
admin> show version;
show_version
+-----------------------+
| version               |
+-----------------------+
| v0.21.0-241-gc6cf58d5 |
+-----------------------+
admin> \q
Goodbye!

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-06 19:24:46 +08:00
Yongteng Lei
0cd8024c34
Feat: RAPTOR handle cancel gracefully (#11074) 
### What problem does this PR solve?

RAPTOR handle cancel gracefully.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-06 17:18:03 +08:00
Kevin Hu
3bd1fefe1f
Feat: debug sync data. (#11073) 
### What problem does this PR solve?

#10953 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-06 16:48:04 +08:00
Yongteng Lei
23b81eae77
Feat: GraphRAG handle cancel gracefully (#11061) 
### What problem does this PR solve?

 GraghRAG handle cancel gracefully. #10997.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-06 16:12:20 +08:00
Kevin Hu
adbb8319e0
Fix: add fields for logs. (#11039) 
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-06 09:49:57 +08:00
Jin Hai
f98b24c9bf
Move api.settings to common.settings (#11036) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-06 09:36:38 +08:00
Kevin Hu
dddf766470
Feat: start data sync service. (#11026) 
### What problem does this PR solve?

#10953 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-05 15:43:15 +08:00
Kevin Hu
b86e07088b
Fix: escape multi-steps issues. (#11016) 
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-11-05 14:51:00 +08:00
Jin Hai
1a9215bc6f
Move some vars to globals (#11017) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 14:14:38 +08:00
Jin Hai
96c015fb85
Fix and refactor imports (#11010) 
### What problem does this PR solve?

1. Move EMBEDDING_CFG to common.globals
2. Fix error imports
3. Move signal handles to common/signal_utils.py

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 11:07:54 +08:00
Jin Hai
bab3fce136
Move some constants to common (#11004) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-05 08:01:39 +08:00
Kevin Hu
4bbbf92331
Refa: link connector to KB. (#10991) 
### What problem does this PR solve?

#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-04 20:13:52 +08:00
Jin Hai
880a6a0428
Move some enumerate type to constants.py (#10998) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 19:25:25 +08:00
Yongteng Lei
465a140727
Feat: refine Confluence connector (#10994) 
### What problem does this PR solve?

Refine Confluence connector.
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
 2025-11-04 17:29:11 +08:00
Jin Hai
16d2be623c
Minor tweaks (#10987) 
### What problem does this PR solve?

1. Rename identifier name
2. Fix some return statement
3. Fix some typos

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 14:15:31 +08:00
Jin Hai
1e45137284
Move 'timeout' to common folder (#10983) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-04 11:51:12 +08:00
Jin Hai
378bdfccfc
Refactor log utils (#10973) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 20:25:02 +08:00
Kevin Hu
3e5a39482e
Feat: Support multiple data sources synchronizations (#10954) 
### What problem does this PR solve?
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-11-03 19:59:18 +08:00
Jin Hai
076d811086
Introduce common/config_utils.py (#10968) 
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 17:25:06 +08:00
Jin Hai
d008a4df9f
Move base64_image related functions to common directory (#10957) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 15:20:46 +08:00
Jin Hai
360f5c1179
Move token related functions to common (#10942) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-03 08:50:05 +08:00
Jin Hai
44f2d6f5da
Move 'get_project_base_directory' to common directory (#10940) 
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-11-02 21:05:28 +08:00
Zhichang Yu
73144e278b
Don't release full image (#10654) 
### What problem does this PR solve?

Introduced gpu profile in .env
Added Dockerfile_tei
fix datrie
Removed LIGHTEN flag

### Type of change

- [x] Documentation Update
- [x] Refactoring
 2025-10-23 23:02:27 +08:00
Yongteng Lei
2d491188b8
Refa: improve flow of GraphRAG and RAPTOR (#10709) 
### What problem does this PR solve?

Improve flow of GraphRAG and RAPTOR.

### Type of change

- [x] Refactoring
 2025-10-22 09:29:20 +08:00
Jin Hai
deb81810e9
Update message printout when start ingestion server (#10677) 
### What problem does this PR solve?

```
     ____                                  __     _                                                                  
    /  _/   ____    ____ _  ___    _____  / /_   (_)  ____    ____           _____  ___    _____ _   __  ___    _____
    / /    / __ \  / __ `/ / _ \  / ___/ / __/  / /  / __ \  / __ \         / ___/ / _ \  / ___/| | / / / _ \  / ___/
 _/ /    / / / / / /_/ / /  __/ (__  ) / /_   / /  / /_/ / / / / /        (__  ) /  __/ / /    | |/ / /  __/ / /    
/___/   /_/ /_/  \__, /  \___/ /____/  \__/  /_/   \____/ /_/ /_/        /____/  \___/ /_/     |___/  \___/ /_/     
                /____/          
```

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-10-21 09:38:20 +08:00
Liu An
8af769de41
Fix: add toc_kwd field and update page_num_int type (#10596) 
### What problem does this PR solve?

- Added new field 'toc_kwd' to infinity_mapping.json for table of
contents keyword support
- Changed page_num_int from integer to array type in task_executor.py to
handle multiple page numbers

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-10-16 12:47:24 +08:00
Kevin Hu
f92a45dcc4
Feat: let toc run asynchronizly... (#10513) 
### What problem does this PR solve?

#10436 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-14 14:14:52 +08:00
Yongteng Lei
65c3f0406c
Fix: maintain backward compatibility for KB tasks (#10508) 
### What problem does this PR solve?

Maintain backward compatibility for KB tasks

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
 2025-10-13 11:53:48 +08:00
Kevin Hu
7d2f65671f
Feat: debugging toc part. (#10486) 
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-11 18:45:21 +08:00
Kevin Hu
0d8791936e
Feat: TOC retrieval (#10456) 
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
 2025-10-10 17:07:55 +08:00
Jin Hai
d931c33ced
Fix typos: retrievaler -> retriever (#10372) 
### What problem does this PR solve?

Fix typos

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2025-10-10 09:17:36 +08:00
Kevin Hu
cbf04ee470
Feat: Use data pipeline to visualize the parsing configuration of the knowledge base (#10423) 
### What problem does this PR solve?

#9869

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: 纷繁下的无奈 <zhileihuang@126.com>
Co-authored-by: huangzl <huangzl@shinemo.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Wilmer <33392318@qq.com>
Co-authored-by: Adrian Weidig <adrianweidig@gmx.net>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: Liu An <asiro@qq.com>
Co-authored-by: buua436 <66937541+buua436@users.noreply.github.com>
Co-authored-by: BadwomanCraZY <511528396@qq.com>
Co-authored-by: cucusenok <31804608+cucusenok@users.noreply.github.com>
Co-authored-by: Russell Valentine <russ@coldstonelabs.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Billy Bao <newyorkupperbay@gmail.com>
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: TensorNull <129579691+TensorNull@users.noreply.github.com>
Co-authored-by: TensorNull <tensor.null@gmail.com>
Co-authored-by: TeslaZY <TeslaZY@outlook.com>
Co-authored-by: Ajay <160579663+aybanda@users.noreply.github.com>
Co-authored-by: AB <aj@Ajays-MacBook-Air.local>
Co-authored-by: 天海蒼灆 <huangaoqin@tecpie.com>
Co-authored-by: He Wang <wanghechn@qq.com>
Co-authored-by: Atsushi Hatakeyama <atu729@icloud.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Mohamed Mathari <155896313+melmathari@users.noreply.github.com>
Co-authored-by: Mohamed Mathari <nocodeventure@Mac-mini-van-Mohamed.fritz.box>
Co-authored-by: Stephen Hu <stephenhu@seismic.com>
Co-authored-by: Shaun Zhang <zhangwfjh@users.noreply.github.com>
Co-authored-by: zhimeng123 <60221886+zhimeng123@users.noreply.github.com>
Co-authored-by: mxc <mxc@example.com>
Co-authored-by: Dominik Novotný <50611433+SgtMarmite@users.noreply.github.com>
Co-authored-by: EVGENY M <168018528+rjohny55@users.noreply.github.com>
Co-authored-by: mcoder6425 <mcoder64@gmail.com>
Co-authored-by: lemsn <lemsn@msn.com>
Co-authored-by: lemsn <lemsn@126.com>
Co-authored-by: Adrian Gora <47756404+adagora@users.noreply.github.com>
Co-authored-by: Womsxd <45663319+Womsxd@users.noreply.github.com>
Co-authored-by: FatMii <39074672+FatMii@users.noreply.github.com>
 2025-10-09 12:36:19 +08:00
Jin Hai
4eb7659499
Fix bug: broken import from rag.prompts.prompts (#10217) 
### What problem does this PR solve?

Fix broken imports

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
 2025-09-23 10:19:25 +08:00