de839fc3f0
### What problem does this PR solve? Optimize task broker and executor for reduce memory usage and deployment complexity. ### Type of change - [x] Performance Improvement - [x] Refactoring ### Change Log - Enhance redis utils for message queue(use stream) - Modify task broker logic via message queue (1.get parse event from message queue 2.use ThreadPoolExecutor async executor ) - Modify the table column name of document and task (process_duation -> process_duration maybe just a spelling mistake) - Reformat some code style(just what i see) - Add requirement_dev.txt for developer - Add redis container on docker compose --------- Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| api_app.py | ||
| chunk_app.py | ||
| conversation_app.py | ||
| dialog_app.py | ||
| document_app.py | ||
| file2document_app.py | ||
| file_app.py | ||
| kb_app.py | ||
| llm_app.py | ||
| user_app.py | ||