2ffe6f7439
### What problem does this PR solve? - Original rag/nlp/rag_tokenizer.py is put to Infinity and infinity-sdk via https://github.com/infiniflow/infinity/pull/3117 . Import rag_tokenizer from infinity and inherit from rag_tokenizer.RagTokenizer in new rag/nlp/rag_tokenizer.py. - Bump infinity to 0.6.8 ### Type of change - [x] Refactoring |
||
|---|---|---|
| .. | ||
| app | ||
| flow | ||
| llm | ||
| nlp | ||
| prompts | ||
| res | ||
| svr | ||
| utils | ||
| __init__.py | ||
| benchmark.py | ||
| raptor.py | ||
| settings.py | ||