gmakstutis/openrag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'main' into issue-673-docs-new-storage-path

Browse source
This commit is contained in:
April M 2025-12-18 12:59:42 -08:00
commit 8811d57203
9 changed files with 1522 additions and 1659 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
.env.example
View file

@ -75,3 +75,10 @@ LANGFLOW_SUPERUSER=
LANGFLOW_SUPERUSER_PASSWORD=
LANGFLOW_NEW_USER_IS_ACTIVE=False
LANGFLOW_ENABLE_SUPERUSER_CLI=False
# Langfuse tracing (optional)
# Get keys from https://cloud.langfuse.com or your self-hosted instance
LANGFUSE_SECRET_KEY=
LANGFUSE_PUBLIC_KEY=
# Leave empty for Langfuse Cloud, or set for self-hosted (e.g., http://localhost:3002)
LANGFUSE_HOST=

4
docker-compose.yml
View file

@ -110,7 +110,9 @@ services:
ports:
- "7860:7860"
environment:
- LANGFLOW_DEACTIVATE_TRACING=true
- LANGFUSE_SECRET_KEY=${LANGFUSE_SECRET_KEY:-}
- LANGFUSE_PUBLIC_KEY=${LANGFUSE_PUBLIC_KEY:-}
- LANGFUSE_HOST=${LANGFUSE_HOST:-}
- OPENAI_API_KEY=${OPENAI_API_KEY:-None}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY:-None}
- WATSONX_API_KEY=${WATSONX_API_KEY:-None}

4
docs/docs/get-started/quickstart.mdx
View file

@ -89,7 +89,7 @@ For example: `What documents are available to you?`
The agent responds with a summary of OpenRAG's default documents.
3. To verify the agent's response, click <Icon name="Library" aria-hidden="true"/> **Knowledge** to view the documents stored in the OpenRAG OpenSearch vector database.
3. To verify the agent's response, click <Icon name="Library" aria-hidden="true"/> **Knowledge** to view the documents stored in the OpenRAG OpenSearch database.
You can click a document to view the chunks of the document as they are stored in the database.
4. Click **Add Knowledge** to add your own documents to your OpenRAG knowledge base.
@ -106,7 +106,7 @@ You can click a document to view the chunks of the document as they are stored i
* Click <Icon name="Gear" aria-hidden="true"/> **Function Call: search_documents (tool_call)** to view the log of tool calls made by the agent. This is helpful for troubleshooting because it shows you how the agent used particular tools.
* Click <Icon name="Library" aria-hidden="true"/> **Knowledge** to confirm that the documents are present in the OpenRAG OpenSearch vector database, and then click each document to see how the document was chunked.
* Click <Icon name="Library" aria-hidden="true"/> **Knowledge** to confirm that the documents are present in the OpenRAG OpenSearch database, and then click each document to see how the document was chunked.
If a document was chunked improperly, you might need to tweak the ingestion or modify and reupload the document.
* Click <Icon name="Settings2" aria-hidden="true"/> **Settings** to modify the knowledge ingestion settings.

8
docs/docs/get-started/what-is-openrag.mdx
View file

@ -4,11 +4,11 @@ slug: /
hide_table_of_contents: true
---
OpenRAG is an open-source package for building agentic RAG systems that integrates with a wide range of orchestration tools, vector databases, and LLM providers.
OpenRAG is an open-source package for building agentic RAG systems that integrates with a wide range of orchestration tools, databases, and LLM providers.
OpenRAG connects and amplifies three popular, proven open-source projects into one powerful platform:
* [Langflow](https://docs.langflow.org): Langflow is a versatile tool for building and deploying AI agents and MCP servers. It supports all major LLMs, vector databases, and a growing library of AI tools.
* [Langflow](https://docs.langflow.org): Langflow is a versatile tool for building and deploying AI agents and MCP servers. It supports all major LLMs, popular vector databases, and a growing library of AI tools.
OpenRAG uses several built-in flows, and it provides full access to all Langflow features through the embedded Langflow visual editor.
@ -17,7 +17,7 @@ OpenRAG connects and amplifies three popular, proven open-source projects into o
* [OpenSearch](https://docs.opensearch.org/latest/): OpenSearch is a community-driven, Apache 2.0-licensed open source search and analytics suite that makes it easy to ingest, search, visualize, and analyze data.
It provides powerful hybrid search capabilities with enterprise-grade security and multi-tenancy support.
OpenRAG uses OpenSearch as the underlying vector database for storing and retrieving your documents and associated vector data (embeddings). You can ingest documents from a variety of sources, including your local filesystem and OAuth authenticated connectors to popular cloud storage services.
OpenRAG uses OpenSearch as the underlying database for storing and retrieving your documents and associated vector data (embeddings). You can ingest documents from a variety of sources, including your local filesystem and OAuth authenticated connectors to popular cloud storage services.
* [Docling](https://docling-project.github.io/docling/): Docling simplifies document processing, supports many file formats and advanced PDF parsing, and provides seamless integrations with the generative AI ecosystem.
@ -59,7 +59,7 @@ flowchart TD
* **OpenRAG backend**: The central orchestration service that coordinates all other components.
* **Langflow**: This container runs a Langflow instance. It provides the embedded Langflow visual editor for editing and creating flow, and it connects to the **OpenSearch** container for vector storage and retrieval.
* **Langflow**: This container runs a Langflow instance. It provides the embedded Langflow visual editor for editing and creating flow, and it connects to the **OpenSearch** container for document storage and retrieval.
* **Docling Serve**: This is a local document processing service managed by the **OpenRAG backend**.

2
pyproject.toml
View file

@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "openrag"
version = "0.1.53"
version = "0.1.54"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13"

26
src/tui/managers/env_manager.py
View file

@ -55,6 +55,11 @@ class EnvConfig:
aws_secret_access_key: str = ""
langflow_public_url: str = ""
# Langfuse settings (optional)
langfuse_secret_key: str = ""
langfuse_public_key: str = ""
langfuse_host: str = ""
# Langflow auth settings
langflow_auto_login: str = "False"
langflow_new_user_is_active: str = "False"
@ -190,6 +195,9 @@ class EnvManager:
"LANGFLOW_ENABLE_SUPERUSER_CLI": "langflow_enable_superuser_cli",
"DISABLE_INGEST_WITH_LANGFLOW": "disable_ingest_with_langflow",
"OPENRAG_VERSION": "openrag_version",
"LANGFUSE_SECRET_KEY": "langfuse_secret_key", # pragma: allowlist secret
"LANGFUSE_PUBLIC_KEY": "langfuse_public_key", # pragma: allowlist secret
"LANGFUSE_HOST": "langfuse_host",
}
loaded_from_file = False
@ -503,6 +511,24 @@ class EnvManager:
if optional_written:
f.write("\n")
# Langfuse settings (optional)
langfuse_vars = [
("LANGFUSE_SECRET_KEY", self.config.langfuse_secret_key),
("LANGFUSE_PUBLIC_KEY", self.config.langfuse_public_key),
("LANGFUSE_HOST", self.config.langfuse_host),
]
langfuse_written = False
for var_name, var_value in langfuse_vars:
if var_value:
if not langfuse_written:
f.write("# Langfuse settings\n")
langfuse_written = True
f.write(f"{var_name}={self._quote_env_value(var_value)}\n")
if langfuse_written:
f.write("\n")
return True
except Exception as e:

64
src/tui/screens/config.py
View file

@ -513,6 +513,58 @@ class ConfigScreen(Screen):
self.inputs["aws_secret_access_key"] = input_widget
yield Static(" ")
# Langfuse Section (available in both basic and full mode)
yield Static("Langfuse (Tracing)", classes="tab-header")
yield Static(" ")
# Langfuse Secret Key
yield Label("Langfuse Secret Key (optional)")
yield Static(
Text("Get keys from your Langfuse project settings", style="dim"),
classes="helper-text",
)
current_value = getattr(self.env_manager.config, "langfuse_secret_key", "")
with Horizontal(id="langfuse-secret-key-row"):
input_widget = Input(
placeholder="sk-lf-...",
value=current_value,
password=True,
id="input-langfuse_secret_key",
)
yield input_widget
self.inputs["langfuse_secret_key"] = input_widget
yield Button("Show", id="toggle-langfuse-secret-key", variant="default")
yield Static(" ")
# Langfuse Public Key
yield Label("Langfuse Public Key (optional)")
current_value = getattr(self.env_manager.config, "langfuse_public_key", "")
with Horizontal(id="langfuse-public-key-row"):
input_widget = Input(
placeholder="pk-lf-...",
value=current_value,
password=True,
id="input-langfuse_public_key",
)
yield input_widget
self.inputs["langfuse_public_key"] = input_widget
yield Button("Show", id="toggle-langfuse-public-key", variant="default")
yield Static(" ")
# Langfuse Base URL
yield Label("Langfuse Host (optional)")
yield Static(
Text("Leave empty for Langfuse Cloud, or set for self-hosted", style="dim"),
classes="helper-text",
)
current_value = getattr(self.env_manager.config, "langfuse_host", "")
input_widget = Input(
placeholder="https://cloud.langfuse.com",
value=current_value,
id="input-langfuse_host",
)
yield input_widget
self.inputs["langfuse_host"] = input_widget
yield Static(" ")
# Other Settings Section
@ -713,6 +765,18 @@ class ConfigScreen(Screen):
if input_widget:
input_widget.password = not input_widget.password
event.button.label = "Hide" if not input_widget.password else "Show"
elif event.button.id == "toggle-langfuse-secret-key":
# Toggle Langfuse secret key visibility
input_widget = self.inputs.get("langfuse_secret_key")
if input_widget:
input_widget.password = not input_widget.password
event.button.label = "Hide" if not input_widget.password else "Show"
elif event.button.id == "toggle-langfuse-public-key":
# Toggle Langfuse public key visibility
input_widget = self.inputs.get("langfuse_public_key")
if input_widget:
input_widget.password = not input_widget.password
event.button.label = "Hide" if not input_widget.password else "Show"
def action_generate(self) -> None:
"""Generate secure passwords for admin accounts."""

45
src/tui/screens/welcome.py
View file

@ -96,7 +96,10 @@ class WelcomeScreen(Screen):
try:
# Use detected runtime command to check services
import subprocess
compose_cmd = self.container_manager.runtime_info.compose_command + ["ps", "--format", "json"]
compose_cmd = self.container_manager.runtime_info.compose_command + [
"-f", str(self.container_manager.compose_file),
"ps", "--format", "json"
]
result = subprocess.run(
compose_cmd,
capture_output=True,
@ -128,20 +131,38 @@ class WelcomeScreen(Screen):
# Check if services are running (exclude starting/created states)
# State can be lowercase or mixed case, so normalize it
running_services = []
starting_services = []
# Only consider expected services (filter out stale/leftover containers)
expected = set(self.container_manager.expected_services)
name_map = self.container_manager.container_name_map
running_services = set()
starting_services = set()
for s in services:
if not isinstance(s, dict):
continue
# Get service name - try compose label first (most reliable for Podman)
labels = s.get('Labels', {}) or {}
service_name = labels.get('com.docker.compose.service', '')
if not service_name:
# Fall back to container name mapping
container_name = s.get('Name') or s.get('Service', '')
if not container_name:
names = s.get('Names', [])
if names and isinstance(names, list):
container_name = names[0]
# Map container name to service name using container_name_map
service_name = name_map.get(container_name, container_name)
# Skip if not an expected service
if service_name not in expected:
continue
state = str(s.get('State', '')).lower()
if state == 'running':
running_services.append(s)
running_services.add(service_name)
elif 'starting' in state or 'created' in state:
starting_services.append(s)
# Only consider services running if we have running services AND no starting services
# This prevents showing the button when containers are still coming up
self.services_running = len(running_services) > 0 and len(starting_services) == 0
starting_services.add(service_name)
# Services are running if all expected services are in running state
# (i.e., we have all expected services running and none are still starting)
self.services_running = len(running_services) == len(expected) and len(starting_services) == 0
else:
self.services_running = False
except Exception:
@ -255,15 +276,15 @@ class WelcomeScreen(Screen):
# Check if services are running
if self.container_manager.is_available():
services = await self.container_manager.get_service_status()
expected = set(self.container_manager.expected_services)
running_services = [
s.name for s in services.values() if s.status == ServiceStatus.RUNNING
]
starting_services = [
s.name for s in services.values() if s.status == ServiceStatus.STARTING
]
# Only consider services running if we have running services AND no starting services
# This prevents showing the button when containers are still coming up
self.services_running = len(running_services) > 0 and len(starting_services) == 0
# Services are running if all expected services are in running state
self.services_running = len(running_services) == len(expected) and len(starting_services) == 0
else:
self.services_running = False

3021
uv.lock generated
View file

File diff suppressed because it is too large Load diff