cherry-pick 3c85e488
This commit is contained in:
Raphaël MANSUY
parent
8b305b47b7
commit
042e19ec7c
2 changed files with 42 additions and 42 deletions
38
README-zh.md
38
README-zh.md
|
|
@ -53,24 +53,24 @@
|
|||
|
||||
## 🎉 新闻
|
||||
|
||||
- [2025.11.05]🎯添加**基于RAGAS的**评估框架和**Langfuse**可观测性支持（API可随查询结果返回召回上下文）。
|
||||
- [2025.10.22]🎯消除处理**大规模数据集**的性能瓶颈。
|
||||
- [2025.09.15]🎯显著提升**小型LLM**（如Qwen3-30B-A3B）的知识图谱提取准确性。
|
||||
- [2025.08.29]🎯现已支持**Reranker**，显著提升混合查询性能(现已设为默认查询模式)。
|
||||
- [2025.08.04]🎯支持**文档删除**并重新生成知识图谱以确保查询性能。
|
||||
- [2025.06.16]🎯我们的团队发布了[RAG-Anything](https://github.com/HKUDS/RAG-Anything)，一个用于无缝处理文本、图像、表格和方程式的全功能多模态 RAG 系统。
|
||||
- [2025.06.05]🎯LightRAG现已集成[RAG-Anything](https://github.com/HKUDS/RAG-Anything)，支持全面的多模态文档解析与RAG能力（PDF、图片、Office文档、表格、公式等）。详见下方[多模态处理模块](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#多模态文档处理rag-anything集成)。
|
||||
- [2025.03.18]🎯LightRAG现已支持参考文献功能。
|
||||
- [2025.02.12]🎯现在您可以使用MongoDB作为一体化存储解决方案。
|
||||
- [2025.02.05]🎯我们团队发布了[VideoRAG](https://github.com/HKUDS/VideoRAG)，用于理解超长上下文视频。
|
||||
- [2025.01.13]🎯我们团队发布了[MiniRAG](https://github.com/HKUDS/MiniRAG)，使用小型模型简化RAG。
|
||||
- [2025.01.06]🎯现在您可以使用PostgreSQL作为一体化存储解决方案。
|
||||
- [2024.11.19]🎯LightRAG的综合指南现已在[LearnOpenCV](https://learnopencv.com/lightrag)上发布。非常感谢博客作者。
|
||||
- [2024.11.09]🎯推出LightRAG Webui，允许您插入、查询、可视化LightRAG知识。
|
||||
- [2024.11.04]🎯现在您可以[使用Neo4J进行存储](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#using-neo4j-for-storage)。
|
||||
- [2024.10.18]🎯我们添加了[LightRAG介绍视频](https://youtu.be/oageL-1I0GE)的链接。感谢作者！
|
||||
- [2024.10.17]🎯我们创建了一个[Discord频道](https://discord.gg/yF2MmDJyGJ)！欢迎加入分享和讨论！🎉🎉
|
||||
- [2024.10.16]🎯LightRAG现在支持[Ollama模型](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#quick-start)！
|
||||
- [x] [2025.11.05]🎯📢添加**基于RAGAS的**评估框架和**Langfuse**可观测性支持（API可随查询结果返回召回上下文）。
|
||||
- [x] [2025.10.22]🎯📢消除处理**大规模数据集**的性能瓶颈。
|
||||
- [x] [2025.09.15]🎯📢显著提升**小型LLM**（如Qwen3-30B-A3B）的知识图谱提取准确性。
|
||||
- [x] [2025.08.29]🎯📢现已支持**Reranker**，显著提升混合查询性能(现已设为默认查询模式)。
|
||||
- [x] [2025.08.04]🎯📢支持**文档删除**并重新生成知识图谱以确保查询性能。
|
||||
- [x] [2025.06.16]🎯📢我们的团队发布了[RAG-Anything](https://github.com/HKUDS/RAG-Anything)，一个用于无缝处理文本、图像、表格和方程式的全功能多模态 RAG 系统。
|
||||
- [x] [2025.06.05]🎯📢LightRAG现已集成[RAG-Anything](https://github.com/HKUDS/RAG-Anything)，支持全面的多模态文档解析与RAG能力（PDF、图片、Office文档、表格、公式等）。详见下方[多模态处理模块](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#多模态文档处理rag-anything集成)。
|
||||
- [x] [2025.03.18]🎯📢LightRAG现已支持参考文献功能。
|
||||
- [x] [2025.02.12]🎯📢现在您可以使用MongoDB作为一体化存储解决方案。
|
||||
- [x] [2025.02.05]🎯📢我们团队发布了[VideoRAG](https://github.com/HKUDS/VideoRAG)，用于理解超长上下文视频。
|
||||
- [x] [2025.01.13]🎯📢我们团队发布了[MiniRAG](https://github.com/HKUDS/MiniRAG)，使用小型模型简化RAG。
|
||||
- [x] [2025.01.06]🎯📢现在您可以使用PostgreSQL作为一体化存储解决方案。
|
||||
- [x] [2024.11.19]🎯📢LightRAG的综合指南现已在[LearnOpenCV](https://learnopencv.com/lightrag)上发布。非常感谢博客作者。
|
||||
- [x] [2024.11.09]🎯📢推出LightRAG Webui，允许您插入、查询、可视化LightRAG知识。
|
||||
- [x] [2024.11.04]🎯📢现在您可以[使用Neo4J进行存储](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#using-neo4j-for-storage)。
|
||||
- [x] [2024.10.18]🎯📢我们添加了[LightRAG介绍视频](https://youtu.be/oageL-1I0GE)的链接。感谢作者！
|
||||
- [x] [2024.10.17]🎯📢我们创建了一个[Discord频道](https://discord.gg/yF2MmDJyGJ)！欢迎加入分享和讨论！🎉🎉
|
||||
- [x] [2024.10.16]🎯📢LightRAG现在支持[Ollama模型](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#quick-start)！
|
||||
|
||||
<details>
|
||||
<summary style="font-size: 1.4em; font-weight: bold; cursor: pointer; display: list-item;">
|
||||
|
|
@ -881,7 +881,7 @@ rag = LightRAG(
|
|||
|
||||
对于生产级场景，您很可能想要利用企业级解决方案。PostgreSQL可以为您提供一站式储解解决方案，作为KV存储、向量数据库（pgvector）和图数据库（apache AGE）。支持 PostgreSQL 版本为16.6或以上。
|
||||
|
||||
* 如果您是初学者并想避免麻烦，推荐使用docker，请从这个镜像开始（默认帐号密码:rag/rag）：https://hub.docker.com/r/gzdaniel/postgres-for-rag
|
||||
* 如果您是初学者并想避免麻烦，推荐使用docker，请从这个镜像开始（请务必阅读概述）：https://hub.docker.com/r/shangor/postgres-for-rag
|
||||
* Apache AGE的性能不如Neo4j。追求高性能的图数据库请使用Noe4j。
|
||||
|
||||
</details>
|
||||
|
|
|
|||
46
README.md
46
README.md
|
|
@ -51,24 +51,24 @@
|
|||
|
||||
---
|
||||
## 🎉 News
|
||||
- [2025.11.05]🎯Add **RAGAS-based** Evaluation Framework and **Langfuse** observability for LightRAG (API can return retrieved contexts with query results).
|
||||
- [2025.10.22]🎯Eliminate bottlenecks in processing **large-scale datasets**.
|
||||
- [2025.09.15]🎯Significantly enhances KG extraction accuracy for **small LLMs** like Qwen3-30B-A3B.
|
||||
- [2025.08.29]🎯**Reranker** is supported now , significantly boosting performance for mixed queries(Set as default query mode now).
|
||||
- [2025.08.04]🎯**Document deletion** with KG regeneration to ensure query performance.
|
||||
- [2025.06.16]🎯Our team has released [RAG-Anything](https://github.com/HKUDS/RAG-Anything) an All-in-One Multimodal RAG System for seamless text, image, table, and equation processing.
|
||||
- [2025.06.05]🎯LightRAG now supports comprehensive multimodal data handling through [RAG-Anything](https://github.com/HKUDS/RAG-Anything) integration, enabling seamless document parsing and RAG capabilities across diverse formats including PDFs, images, Office documents, tables, and formulas. Please refer to the new [multimodal section](https://github.com/HKUDS/LightRAG/?tab=readme-ov-file#multimodal-document-processing-rag-anything-integration) for details.
|
||||
- [2025.03.18]🎯LightRAG now supports citation functionality, enabling proper source attribution.
|
||||
- [2025.02.12]🎯You can now use MongoDB as all in-one Storage.
|
||||
- [2025.02.05]🎯Our team has released [VideoRAG](https://github.com/HKUDS/VideoRAG) understanding extremely long-context videos.
|
||||
- [2025.01.13]🎯Our team has released [MiniRAG](https://github.com/HKUDS/MiniRAG) making RAG simpler with small models.
|
||||
- [2025.01.06]🎯You can now use PostgreSQL as all in-one Storage.
|
||||
- [2024.11.19]🎯A comprehensive guide to LightRAG is now available on [LearnOpenCV](https://learnopencv.com/lightrag). Many thanks to the blog author.
|
||||
- [2024.11.09]🎯Introducing the LightRAG Webui, which allows you to insert, query, visualize LightRAG knowledge.
|
||||
- [2024.11.04]🎯You can now [use Neo4J for Storage](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#using-neo4j-for-storage).
|
||||
- [2024.10.18]🎯We've added a link to a [LightRAG Introduction Video](https://youtu.be/oageL-1I0GE). Thanks to the author!
|
||||
- [2024.10.17]🎯We have created a [Discord channel](https://discord.gg/yF2MmDJyGJ)! Welcome to join for sharing and discussions! 🎉🎉
|
||||
- [2024.10.16]🎯LightRAG now supports [Ollama models](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#quick-start)!
|
||||
- [x] [2025.11.05]🎯📢Add **RAGAS-based** Evaluation Framework and **Langfuse** observability for LightRAG (API can return retrieved contexts with query results).
|
||||
- [x] [2025.10.22]🎯📢Eliminate bottlenecks in processing **large-scale datasets**.
|
||||
- [x] [2025.09.15]🎯📢Significantly enhances KG extraction accuracy for **small LLMs** like Qwen3-30B-A3B.
|
||||
- [x] [2025.08.29]🎯📢**Reranker** is supported now , significantly boosting performance for mixed queries(Set as default query mode now).
|
||||
- [x] [2025.08.04]🎯📢**Document deletion** with KG regeneration to ensure query performance.
|
||||
- [x] [2025.06.16]🎯📢Our team has released [RAG-Anything](https://github.com/HKUDS/RAG-Anything) an All-in-One Multimodal RAG System for seamless text, image, table, and equation processing.
|
||||
- [x] [2025.06.05]🎯📢LightRAG now supports comprehensive multimodal data handling through [RAG-Anything](https://github.com/HKUDS/RAG-Anything) integration, enabling seamless document parsing and RAG capabilities across diverse formats including PDFs, images, Office documents, tables, and formulas. Please refer to the new [multimodal section](https://github.com/HKUDS/LightRAG/?tab=readme-ov-file#multimodal-document-processing-rag-anything-integration) for details.
|
||||
- [x] [2025.03.18]🎯📢LightRAG now supports citation functionality, enabling proper source attribution.
|
||||
- [x] [2025.02.12]🎯📢You can now use MongoDB as all in-one Storage.
|
||||
- [x] [2025.02.05]🎯📢Our team has released [VideoRAG](https://github.com/HKUDS/VideoRAG) understanding extremely long-context videos.
|
||||
- [x] [2025.01.13]🎯📢Our team has released [MiniRAG](https://github.com/HKUDS/MiniRAG) making RAG simpler with small models.
|
||||
- [x] [2025.01.06]🎯📢You can now use PostgreSQL as all in-one Storage.
|
||||
- [x] [2024.11.19]🎯📢A comprehensive guide to LightRAG is now available on [LearnOpenCV](https://learnopencv.com/lightrag). Many thanks to the blog author.
|
||||
- [x] [2024.11.09]🎯📢Introducing the LightRAG Webui, which allows you to insert, query, visualize LightRAG knowledge.
|
||||
- [x] [2024.11.04]🎯📢You can now [use Neo4J for Storage](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#using-neo4j-for-storage).
|
||||
- [x] [2024.10.18]🎯📢We've added a link to a [LightRAG Introduction Video](https://youtu.be/oageL-1I0GE). Thanks to the author!
|
||||
- [x] [2024.10.17]🎯📢We have created a [Discord channel](https://discord.gg/yF2MmDJyGJ)! Welcome to join for sharing and discussions! 🎉🎉
|
||||
- [x] [2024.10.16]🎯📢LightRAG now supports [Ollama models](https://github.com/HKUDS/LightRAG?tab=readme-ov-file#quick-start)!
|
||||
|
||||
<details>
|
||||
<summary style="font-size: 1.4em; font-weight: bold; cursor: pointer; display: list-item;">
|
||||
|
|
@ -214,7 +214,7 @@ For a streaming response implementation example, please see `examples/lightrag_o
|
|||
|
||||
**Note 2**: Only `lightrag_openai_demo.py` and `lightrag_openai_compatible_demo.py` are officially supported sample codes. Other sample files are community contributions that haven't undergone full testing and optimization.
|
||||
|
||||
## Programming with LightRAG Core
|
||||
## Programing with LightRAG Core
|
||||
|
||||
> ⚠️ **If you would like to integrate LightRAG into your project, we recommend utilizing the REST API provided by the LightRAG Server**. LightRAG Core is typically intended for embedded applications or for researchers who wish to conduct studies and evaluations.
|
||||
|
||||
|
|
@ -313,7 +313,7 @@ A full list of LightRAG init parameters:
|
|||
| **vector_db_storage_cls_kwargs** | `dict` | Additional parameters for vector database, like setting the threshold for nodes and relations retrieval | cosine_better_than_threshold: 0.2（default value changed by env var COSINE_THRESHOLD) |
|
||||
| **enable_llm_cache** | `bool` | If `TRUE`, stores LLM results in cache; repeated prompts return cached responses | `TRUE` |
|
||||
| **enable_llm_cache_for_entity_extract** | `bool` | If `TRUE`, stores LLM results in cache for entity extraction; Good for beginners to debug your application | `TRUE` |
|
||||
| **addon_params** | `dict` | Additional parameters, e.g., `{"language": "Simplified Chinese", "entity_types": ["organization", "person", "location", "event"]}`: sets example limit, entity/relation extraction output language | language: English` |
|
||||
| **addon_params** | `dict` | Additional parameters, e.g., `{"language": "Simplified Chinese", "entity_types": ["organization", "person", "location", "event"]}`: sets example limit, entiy/relation extraction output language | language: English` |
|
||||
| **embedding_cache_config** | `dict` | Configuration for question-answer caching. Contains three parameters: `enabled`: Boolean value to enable/disable cache lookup functionality. When enabled, the system will check cached responses before generating new answers. `similarity_threshold`: Float value (0-1), similarity threshold. When a new question's similarity with a cached question exceeds this threshold, the cached answer will be returned directly without calling the LLM. `use_llm_check`: Boolean value to enable/disable LLM similarity verification. When enabled, LLM will be used as a secondary check to verify the similarity between questions before returning cached answers. | Default: `{"enabled": False, "similarity_threshold": 0.95, "use_llm_check": False}` |
|
||||
|
||||
</details>
|
||||
|
|
@ -364,7 +364,7 @@ class QueryParam:
|
|||
max_total_tokens: int = int(os.getenv("MAX_TOTAL_TOKENS", "30000"))
|
||||
"""Maximum total tokens budget for the entire query context (entities + relations + chunks + system prompt)."""
|
||||
|
||||
# History messages are only sent to LLM for context, not used for retrieval
|
||||
# History mesages is only send to LLM for context, not used for retrieval
|
||||
conversation_history: list[dict[str, str]] = field(default_factory=list)
|
||||
"""Stores past conversation history to maintain context.
|
||||
Format: [{"role": "user/assistant", "content": "message"}].
|
||||
|
|
@ -845,7 +845,7 @@ see test_neo4j.py for a working example.
|
|||
For production level scenarios you will most likely want to leverage an enterprise solution. PostgreSQL can provide a one-stop solution for you as KV store, VectorDB (pgvector) and GraphDB (apache AGE). PostgreSQL version 16.6 or higher is supported.
|
||||
|
||||
* PostgreSQL is lightweight,the whole binary distribution including all necessary plugins can be zipped to 40MB: Ref to [Windows Release](https://github.com/ShanGor/apache-age-windows/releases/tag/PG17%2Fv1.5.0-rc0) as it is easy to install for Linux/Mac.
|
||||
* If you prefer docker, please start with this image if you are a beginner to avoid hiccups (Default user password:rag/rag): https://hub.docker.com/r/gzdaniel/postgres-for-rag
|
||||
* If you prefer docker, please start with this image if you are a beginner to avoid hiccups (DO read the overview): https://hub.docker.com/r/shangor/postgres-for-rag
|
||||
* How to start? Ref to: [examples/lightrag_zhipu_postgres_demo.py](https://github.com/HKUDS/LightRAG/blob/main/examples/lightrag_zhipu_postgres_demo.py)
|
||||
* For high-performance graph database requirements, Neo4j is recommended as Apache AGE's performance is not as competitive.
|
||||
|
||||
|
|
@ -1555,7 +1555,7 @@ Langfuse provides a drop-in replacement for the OpenAI client that automatically
|
|||
pip install lightrag-hku
|
||||
pip install lightrag-hku[observability]
|
||||
|
||||
# Or install from source code with debug mode enabled
|
||||
# Or install from souce code with debug mode enabled
|
||||
pip install -e .
|
||||
pip install -e ".[observability]"
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue