Commit graph

3875 commits

Author SHA1 Message Date
Taddeus
2255b91210 Adds initial LightRAG app integration with schema and processors
Introduces the LightRAG Retrieval-Augmented Generation framework as an Apolo app, including input/output schemas, types, and processors.
Adds Helm chart value processing, environment and persistence configurations, and output service discovery for deployment.
Includes scripts for generating type schemas and testing support, along with CI and linting setup tailored for the new app.
Provides a documentation loader script to ingest markdown files into LightRAG with flexible referencing modes.

Relates to MLO-469
2025-11-03 13:59:52 +02:00
Taddeus
748ded40fb
MLO-446: Adds API key authentication support to LightRAG client (#12)
* Adds LightRAG API key support to deployment and secrets

Introduces a new environment variable for the LightRAG API key sourced from secrets to enable authenticated access.

Updates Helm values and templates to include LightRAG API key management alongside the existing OpenAI key, improving configuration consistency and security.

Relates to MLO-339

* Adds optional API key authentication support to LightRAG client

Enables passing custom headers, including an API key from environment variables, to all LightRAG HTTP requests for authentication.

Improves security by allowing authenticated access without breaking existing unauthenticated usage.

Relates to MLO-446

* Adds basic user authentication support to Helm deployment

Introduces configurable user accounts and token secret in values and templates to enable authentication.

Generates an encoded authentication string from account data stored in secrets and exposes relevant environment variables in the deployment only when authentication is enabled and configured.

This enhancement allows secure management of multiple user credentials and token secrets, improving the deployment's security and flexibility.

Relates to MLO-446

* Adds support for external secret references in PostgreSQL auth

Introduces parameters to allow PostgreSQL credentials to be sourced from existing Kubernetes secrets instead of inline passwords.

Improves security and flexibility by enabling integration with external secret management without changing deployment structure.

Relates to MLO-446

* Streamline deployment docs and remove preset environment configs

Consolidates deployment instructions by removing separate dev and prod values files and related workflows, encouraging users to customize a single values file instead.

Simplifies the README to focus on flexible chart deployment without environment-specific templates or variable substitution, improving maintainability and clarity.

* Adds Helm packaging and publishing Makefile for LightRAG

Introduces a Makefile to automate Helm chart packaging, versioning, and publishing to a container registry.

Uses git tags or user-defined versions for chart versioning with sanitization.

Ensures streamlined CI/CD by handling dependencies, packaging, registry login, and cleanup, simplifying release workflows.

Relates to MLO-446
2025-10-29 14:31:56 +02:00
Taddeus
aabe4aa434
MLO-340: secret fixes (#11)
* Updates LLM and embedding configurations to use OpenRouter and Gemini

* Renames and significantly expands environment configuration template

Renames the environment example file to a standard hidden env template to align with common conventions.

Extensively updates and reorganizes configuration options, adding detailed setup for LLM, embedding, storage backends, PostgreSQL, and overall LightRAG processing parameters.

Comments out some legacy and optional configuration lines to streamline initial setup and clarify default recommended values.

Updates gitignore to exclude various env-related files to protect sensitive keys and improve environment management.

* Updates default config with improved LLM and processing settings

* Adds openai-compatible environment file to .gitignore

* Adds new environment files to ignore list

* Refactor API key handling and PostgreSQL host naming
2025-07-28 11:05:16 +03:00
Taddeus
f494b96a7b
MLO-340: Update lightrag settings (#10)
* Updates LLM and embedding configurations to use OpenRouter and Gemini

* Renames and significantly expands environment configuration template

Renames the environment example file to a standard hidden env template to align with common conventions.

Extensively updates and reorganizes configuration options, adding detailed setup for LLM, embedding, storage backends, PostgreSQL, and overall LightRAG processing parameters.

Comments out some legacy and optional configuration lines to streamline initial setup and clarify default recommended values.

Updates gitignore to exclude various env-related files to protect sensitive keys and improve environment management.

* Updates default config with improved LLM and processing settings

* Adds openai-compatible environment file to .gitignore

* Adds new environment files to ignore list
2025-07-24 17:49:50 +03:00
Taddeus
e2a894ee5e
Updates LLM and embedding configurations to use OpenRouter and Gemini (#8) 2025-07-18 17:19:33 +03:00
Taddeus
8c83a97b8e
Adds support for pod labels, tolerations, affinity, and node selectors (#7)
Introduces a template helper for injecting custom pod labels from values.

Enables configuring tolerations, affinity, and node selectors in the deployment spec via values to improve scheduling flexibility and control.

Enhances pod labeling and scheduling customization without hardcoding, supporting more dynamic deployments.
2025-07-09 09:58:30 +03:00
Taddeus
bee93af664
Removes forward authentication middleware and related config (#6) 2025-07-04 16:06:20 +03:00
Taddeus
148a764d38
Phase 1: Pin LightRAG version to 1.3.8 as new version introduces bug (#5)
* Partial implementation of phase-0

* Partial implementation of phase-1

* add report

* add postgress

* Revert "add postgress"

This reverts commit 27778dc6bb3906b5220dd386e47fe32ca7415332.

* remove junk

* Cleaned up annd setup docs

* update docs

* moved report

* Updated load_markdown_files function: Now returns tuples with (content, title, relative_path) instead of just (content, title)

* fixes to load docs script and more env variables for llm configuration

* update prod values

* update docs

* apolo docs support with linking

* update docs to reflect url conventions and mapping with docs

* Adds ingress and forwardAuth configurations

Adds ingress configuration to expose the application.

Adds forwardAuth configuration to enable user authentication.

Includes middleware to strip headers.

* Adds ingress and forwardAuth configurations

Adds ingress configuration to expose the application.

Adds forwardAuth configuration to enable user authentication.

Includes middleware to strip headers.

* Adds ingress and forward authentication middleware support

* Adds ingress and forward authentication middleware support

* Enables vector extension initialization in PostgreSQL container

Mounts initialization scripts in the PostgreSQL service to automate enabling the vector extension on startup.

This ensures the vector extension is available without manual setup, improving developer experience and consistency across environments.

* Pins application image version to 1.3.8
2025-06-30 18:05:32 +03:00
Taddeus
a70ba1f75a
Phase 1: LightRAG Minimal Helm chart and documentation indexing using url references (#2)
* Partial implementation of phase-0

* Partial implementation of phase-1

* add report

* add postgress

* Revert "add postgress"

This reverts commit 27778dc6bb3906b5220dd386e47fe32ca7415332.

* remove junk

* Cleaned up annd setup docs

* update docs

* moved report

* Updated load_markdown_files function: Now returns tuples with (content, title, relative_path) instead of just (content, title)

* fixes to load docs script and more env variables for llm configuration

* update prod values

* update docs

* apolo docs support with linking

* update docs to reflect url conventions and mapping with docs

* Adds ingress and forwardAuth configurations

Adds ingress configuration to expose the application.

Adds forwardAuth configuration to enable user authentication.

Includes middleware to strip headers.

* Adds ingress and forward authentication middleware support
2025-06-23 20:04:34 +03:00
Taddeus
4a049bc84f
Merge pull request #3 from HKUDS/main
Latest updates
2025-06-17 16:47:48 +03:00
chaohuang-ai
a408465602
Update README.md 2025-06-17 10:39:30 +08:00
chaohuang-ai
caf0411889
Update README.md 2025-06-17 10:38:34 +08:00
zrguo
eea64b9763
Merge pull request #1688 from HKUDS/RAGAnything
RAG-Anything Integration
2025-06-17 01:23:07 +08:00
zrguo
03dd99912d RAG-Anything Integration 2025-06-17 01:16:02 +08:00
yangdx
1389265695 Bump api version to 1.3.9 2025-06-11 17:03:27 +08:00
Daniel.y
2639c03634
Merge pull request #1673 from danielaskdd/litellm-problem
feat: Support `application/octet-stream` requests from LiteLLM clients for Ollama Emulation
2025-06-11 17:02:14 +08:00
yangdx
62621979f0 Bump api version to 0173 2025-06-11 16:37:52 +08:00
yangdx
888be97b06 Fix linting 2025-06-11 16:36:05 +08:00
yangdx
ba5e95ea9e Merge branch 'main' into litellm-problem 2025-06-11 16:27:32 +08:00
yangdx
c3f5c413fa feat(api): add /ps endpoint to list running models for Ollama API 2025-06-11 16:23:02 +08:00
yangdx
9351b09cc7 Enhance Ollama interface compatibility 2025-06-11 15:16:32 +08:00
yangdx
7b07d4c917 feat: 支持 LiteLLM 客户端的 application/octet-stream 请求
修改 Ollama API 路由处理程序,使 /chat 和 /generate 端点能够接受 Content-Type 为 application/octet-stream 的请求。通过绕过 FastAPI 的自动请求验证机制,手动解析请求体,解决了 LiteLLM 客户端连接时出现的 422 错误。此更改保持了对现有 application/json 请求的向后兼容性。
2025-06-11 13:42:30 +08:00
yangdx
ad81e59d9a Fix logger problem 2025-06-11 13:30:45 +08:00
zrguo
5ccdbd6de8
Merge pull request #1667 from kwilt/typo-fix
fix typo: "extrat" -> extract
2025-06-10 11:53:04 +08:00
kwilt
09cbcc4572 fix typo: "extrat" -> extract 2025-06-09 08:28:14 -05:00
zrguo
ea2fabe6b0
Merge pull request #1619 from earayu/add_doc_for_parall
Add doc for explaining LightRAG's multi-document concurrent processing mechanism
2025-06-09 09:50:41 +08:00
zrguo
75d13cc387 fix lint 2025-06-09 09:11:50 +08:00
zrguo
16ef318366
Merge pull request #1665 from neno-is-ooo/fix-initialization-documentation
docs: Add clear initialization requirements and troubleshooting section
2025-06-09 09:10:54 +08:00
neno-is-ooo
199869f45c docs: Add clear initialization requirements and troubleshooting section
- Add prominent warning about required initialization steps
- Document common errors (AttributeError: __aenter__ and KeyError: 'history_messages')
- Add troubleshooting section with specific solutions
- Add inline comments in code example highlighting initialization requirements

This addresses user confusion when LightRAG fails with cryptic errors due to
missing initialization calls. The documentation now clearly states that both
await rag.initialize_storages() and await initialize_pipeline_status() must
be called after creating a LightRAG instance.
2025-06-08 12:43:17 +02:00
chaohuang-ai
20c05a7e77
Update README.md 2025-06-07 00:58:56 +08:00
chaohuang-ai
5b7129f582
Update README.md 2025-06-06 22:16:27 +08:00
zrguo
dc97b2b84f Update README.md 2025-06-05 17:51:04 +08:00
zrguo
3325f20507
Merge pull request #1659 from HKUDS/mineru_integration
MinerU integration
2025-06-05 17:43:27 +08:00
zrguo
cc9040d70c fix lint 2025-06-05 17:37:11 +08:00
zrguo
962974589a Add example of directly using modal processors 2025-06-05 17:36:05 +08:00
zrguo
8a726f6e08 MinerU integration 2025-06-05 17:02:48 +08:00
Daniel.y
5ec81d652d
Merge pull request #1647 from HKUDS/LaTex-support
feat: add LaTeX formula rendering support to ChatMessage component
2025-06-02 18:19:58 +08:00
yangdx
1575c7a5f4 Update webui assets 2025-06-02 18:11:04 +08:00
yangdx
b7a60a5895 fix: use dynamic imports for KaTeX to resolve module initialization error
- Prevents circular dependency issues that caused "Cannot accexss 'gm' before initialization" errors when rendering LaTeX formulas.
2025-06-02 18:10:53 +08:00
yangdx
ca3ae05356 Update webui assets 2025-06-02 17:46:22 +08:00
yangdx
66e4109cff Bump api version to 0172 2025-06-02 17:44:44 +08:00
yangdx
7f50a11977 feat: add LaTeX formula rendering support to ChatMessage component
- Add KaTeX dependencies (katex, rehype-katex, @types/katex)
- Configure ReactMarkdown with rehype-katex plugin
- Support both inline ($...$) and block ($$...$$) LaTeX formulas
- Add theme-aware error handling and responsive styling
2025-06-02 17:44:10 +08:00
yangdx
d94b5f07a4 Update .gitignore 2025-06-02 17:26:00 +08:00
yangdx
36a736db0b Fix node merge error 2025-05-30 12:30:24 +08:00
zrguo
c9a6256af8 Move batch_eval.py to ./reproduce/ 2025-05-27 16:09:57 +08:00
zrguo
40b10e8fcf Update insert_custom_kg 2025-05-27 16:07:04 +08:00
zrguo
5385616e7e
Merge pull request #1633 from omriAl/citation-support-in-custom-graph-creation
Adding citation support in custom graph creation
2025-05-27 15:11:26 +08:00
omri.alon
efccdf0838 Adding citation support in custom graph creation 2025-05-26 20:30:59 +03:00
yangdx
070a5db801 Update README 2025-05-23 12:50:48 +08:00
earayu
2679f619b6 feat: add doc 2025-05-23 11:57:45 +08:00