Commit graph

4317 commits

Author SHA1 Message Date
Jin Hai
96c015fb85
Fix and refactor imports (#11010)
### What problem does this PR solve?

1. Move EMBEDDING_CFG to common.globals
2. Fix error imports
3. Move signal handles to common/signal_utils.py

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-05 11:07:54 +08:00
buua436
ca40b56839
Feat:Data Operations (#11002)
### What problem does this PR solve?

new component:Data Operations

#10427

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-05 10:49:41 +08:00
Wanderson Pinto dos Santos
3654ae61c1
feat: add allowed factories variable to allow admins to restrict llms users can add (#11003)
### What problem does this PR solve?

Currently, if we want to restrict the allowed factories users can use we
need to delete from the database table manually. The proposal of this PR
is to include a variable to that, if set, will restrict the LLM
factories the users can see and add. This allow us to not touch the
llm_factories.json or the database if the LLM factory is already
inserted.

Obs.: All the lint changes were from the pre-commit hook which I did not
change.

### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-11-05 10:47:50 +08:00
Jin Hai
bab3fce136
Move some constants to common (#11004)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-05 08:01:39 +08:00
Kevin Hu
4bbbf92331
Refa: link connector to KB. (#10991)
### What problem does this PR solve?

#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-04 20:13:52 +08:00
balibabu
db9fa3042b
Feat: Add a form with data operations operators #10427 (#11001)
### What problem does this PR solve?

Feat: Add a form with data operations operators #10427

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-04 19:42:59 +08:00
Jin Hai
880a6a0428
Move some enumerate type to constants.py (#10998)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-04 19:25:25 +08:00
Yongteng Lei
465a140727
Feat: refine Confluence connector (#10994)
### What problem does this PR solve?

Refine Confluence connector.
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2025-11-04 17:29:11 +08:00
Yongteng Lei
2677617f93
Feat: supports MinerU http-client/server method (#10961)
### What problem does this PR solve?

Add support for MinerU http-client/server method.

To use MinerU with vLLM server:

1. Set up a vLLM server running MinerU:
   ```bash
   mineru-vllm-server --port 30000
   ```

2. Configure the following environment variables:
- `MINERU_EXECUTABLE=/ragflow/uv_tools/.venv/bin/mineru` (or the path to
your MinerU executable)
   - `MINERU_BACKEND="vlm-http-client"`
   - `MINERU_SERVER_URL="http://your-vllm-server-ip:30000"`

3. Follow the standard MinerU setup steps as described above.

With this configuration, RAGFlow will connect to your vLLM server to
perform document parsing, which can significantly improve parsing
performance for complex documents while reducing the resource
requirements on your RAGFlow server.



![1](https://github.com/user-attachments/assets/46624a0c-0f3b-423e-ace8-81801e97a27d)

![2](https://github.com/user-attachments/assets/66ccc004-a598-47d4-93cb-fe176834f83b)


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2025-11-04 16:03:30 +08:00
Jin Hai
03038c7d3d
Update RetCode to common.constants (#10984)
### What problem does this PR solve?

1. Update RetCode to common.constants
2. Decouple the admin and API modules

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-04 15:12:53 +08:00
Jin Hai
16d2be623c
Minor tweaks (#10987)
### What problem does this PR solve?

1. Rename identifier name
2. Fix some return statement
3. Fix some typos

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-04 14:15:31 +08:00
balibabu
021b2ac51a
Feat: Add data operation node #10427 (#10985)
### What problem does this PR solve?

Feat: Add data operation node #10427

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-04 13:48:44 +08:00
Billy Bao
19f71a961a
Fix: Create dataset performance unmatched between HTTP api and web ui (#10960)
### What problem does this PR solve?

Fix: Create dataset performance unmatched between HTTP api and web ui
#10925

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-04 13:45:14 +08:00
Jin Hai
1e45137284
Move 'timeout' to common folder (#10983)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-04 11:51:12 +08:00
buua436
5283a10387
Fix:wrong param in meta_data_filter (#10978)
### What problem does this PR solve?
change:
wrong param in meta_data_filter

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-04 11:22:10 +08:00
Jin Hai
d55344bc11
Remove unused code (#10981)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-04 11:10:29 +08:00
Liu An
640e8e3f3e
Chore(docker): Remove outdated sandbox config (#10977)
### What problem does this PR solve?

Remove outdated sandbox config

### Type of change

- [x] Refactoring
2025-11-04 10:59:56 +08:00
Stephen Hu
c20f5675c6
Fix: elasticsearch connection hardcoded (#10975)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/10930

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-04 10:59:35 +08:00
Jin Hai
378bdfccfc
Refactor log utils (#10973)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 20:25:02 +08:00
Marvion
395ce16b3c
Fix: correct MCP server authentication header format in frontend (#9819)
- Fix MCP test connection authentication issues by updating frontend
request format
- Add variables field with authorization_token for template substitution
- Change headers to use proper Authorization Bearer format with template
variable

🤖 Generated with [Claude Code](https://claude.ai/code)

### What problem does this PR solve?

correct MCP server authentication header format in frontend
### Type of change

 * [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Marvion <marvionliu@wukongjx.cn>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-03 20:00:27 +08:00
balibabu
be3ae0eda9
Feat: Add variables to the metadata filtering function of the knowledge retrieval component. #10861 (#10974)
### What problem does this PR solve?

Feat: Add variables to the metadata filtering function of the knowledge
retrieval component. #10861

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 19:59:45 +08:00
Kevin Hu
3e5a39482e
Feat: Support multiple data sources synchronizations (#10954)
### What problem does this PR solve?
#10953

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 19:59:18 +08:00
Jin Hai
9a486e0f51
Move some funcs from api to rag module (#10972)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 19:26:09 +08:00
balibabu
ee9ac15174
Feat: Fixed an issue where dragged operators within an iteration were not associated with the iteration. #10866 (#10969)
### What problem does this PR solve?

Feat: Fixed an issue where dragged operators within an iteration were
not associated with the iteration. #10866

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 19:19:26 +08:00
buua436
ac465ba2a6
Feat:add variables to the metadata filtering function of the knowledg… (#10967)
…e retrieval component.

### What problem does this PR solve?

issue:
#10861 
change:
add variables to the metadata filtering function of the knowledge
retrieval component

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 19:19:09 +08:00
buua436
fd4aa79c07
Fix:missing embedding vector on Tokenizer (#10964)
### What problem does this PR solve?
issue:
[#10890](https://github.com/infiniflow/ragflow/issues/10890)
change:
missing embedding vector on Tokenizer
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-03 19:17:05 +08:00
buua436
2d83c64eed
Fix:wrong describe_with_prompt() in ollama (#10963)
### What problem does this PR solve?

change:
wrong describe_with_prompt() in ollama

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-03 19:16:41 +08:00
Jin Hai
1284647694
Refactor file utils (#10970)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 18:54:55 +08:00
Jin Hai
076d811086
Introduce common/config_utils.py (#10968)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 17:25:06 +08:00
Jin Hai
121d3fd815
Introduce common/constants.py (#10965)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 16:32:37 +08:00
Jin Hai
d008a4df9f
Move base64_image related functions to common directory (#10957)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 15:20:46 +08:00
balibabu
5a88c01111
Feat: Filter structured output data directly during the rendering stage. #10866 (#10958)
### What problem does this PR solve?

Feat: Filter structured output data directly during the rendering stage.
#10866

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 14:48:35 +08:00
Jin Hai
256b0fb19c
Remove redundant ut (#10955)
### What problem does this PR solve?

Remove redundant ut cases.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 13:04:20 +08:00
Jin Hai
78631a3fd3
Move some functions out of 'api/utils/common.py' (#10948)
### What problem does this PR solve?

as title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 12:34:47 +08:00
Yongteng Lei
4117f41758
Fix: decode error in email parser app (#10920)
### What problem does this PR solve?

Fix: UnicodeDecodeError: 'gb2312' codec can't decode byte 0xab in
position 560: illegal multibyte sequence.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-03 12:31:06 +08:00
balibabu
a52bdf0b7e
Feat: The structured output of the variable query can also be clicked. #10866 (#10952)
### What problem does this PR solve?

Feat: The structured output of the variable query can also be clicked.
#10866

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 12:30:30 +08:00
Stephen Hu
b47361432a
Fix: API: chunk.update does not update positions (#10945)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/10944

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-03 11:01:44 +08:00
Lynn
061d8f78e5
Feat: location rule for http (#10901)
### What problem does this PR solve?

Location rule for http.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 11:01:24 +08:00
Jimmy Ben Klieve
7ec587fa9e
Feat: Admin UI whitelist management and role management (#10910)
### What problem does this PR solve?

Add whitelist management and role management in Admin UI

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 09:52:23 +08:00
Carlo
685311814f
description (#10928)
### Type of change

- [x] Documentation Update
2025-11-03 09:50:21 +08:00
balibabu
410c0a829d
Feat: The query variable of a loop operator can be a nested array variable. #10866 (#10921)
### What problem does this PR solve?

Feat: The query variable of a loop operator can be a nested array
variable. #10866

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 09:40:47 +08:00
buua436
33371cda11
Fix:output_structure in agent (#10907)
### What problem does this PR solve?
change:
output_structure in agent

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-03 09:39:53 +08:00
Billy Bao
fa210e7c58
Feat: parsing hyperlinks in docx and pdf & Fix: default parser config of toc extraction (#10877)
### What problem does this PR solve?

Feat: parsing hyperlinks in docx and pdf #10848
Fix: default parser config of toc extraction

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-11-03 09:34:12 +08:00
Jin Hai
360f5c1179
Move token related functions to common (#10942)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 08:50:05 +08:00
Jin Hai
44f2d6f5da
Move 'get_project_base_directory' to common directory (#10940)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-02 21:05:28 +08:00
Jin Hai
57a83eca8a
Remove unused code (#10938)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-02 16:25:16 +08:00
Jin Hai
6447b737ab
Move singleton to common directory (#10935)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-02 12:24:08 +08:00
Zhichang Yu
fe4852cb71
TEI auto truncate inputs (#10916)
### What problem does this PR solve?

TEI auto truncate inputs

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-31 16:46:20 +08:00
Jin Hai
f52e56c2d6
Remove 'get_lan_ip' and add common misc_utils.py (#10880)
### What problem does this PR solve?

Add get_uuid, download_img and hash_str2int into misc_utils.py

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-31 16:42:01 +08:00
balibabu
e9debfd74d
Fix: The nodes on the canvas were not updated in time after the operator name was modified. #10866 (#10911)
### What problem does this PR solve?

Fix: The nodes on the canvas were not updated in time after the operator
name was modified. #10866

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-31 14:46:03 +08:00