more-on-local-models

This commit is contained in:
Mendon Kissling 2025-09-26 17:11:14 -04:00
parent bc9f181abd
commit 18a41f2ead
2 changed files with 41 additions and 5 deletions

View file

@ -83,11 +83,12 @@ For more information on virtual environments, see [uv](https://docs.astral.sh/uv
The application opens. The application opens.
8. Select your language model and embedding model provider, and complete the required fields. 8. Select your language model and embedding model provider, and complete the required fields.
**Your provider can only be selected once, and you must use the same provider for your language model and embedding model. **Your provider can only be selected once, and you must use the same provider for your language model and embedding model.
To change your selection, you must restart OpenRAG.** To change your provider selection, you must restart OpenRAG and delete the `config.yml` file.**
**Your language model can be changed, but your embeddings model cannot be changed.**
<Tabs groupId="Embedding provider"> <Tabs groupId="Embedding provider">
<TabItem value="OpenAI" label="OpenAI" default> <TabItem value="OpenAI" label="OpenAI" default>
9. You already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, so enable **Get API key from environment variable**. 9. If you already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, enable **Get API key from environment variable**.
10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**. 10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**.
11. To load 2 sample PDFs, enable **Sample dataset**. 11. To load 2 sample PDFs, enable **Sample dataset**.
This is recommended, but not required. This is recommended, but not required.

View file

@ -12,7 +12,40 @@ Get started with OpenRAG by loading your knowledge, swapping out your language m
## Prerequisites ## Prerequisites
- [Install and start OpenRAG](/install) - [Install and start OpenRAG](/install)
- [Langflow API key](/) - Create a [Langflow API key](https://docs.langflow.org/api-keys-and-authentication)
<details>
<summary>Create a Langflow API key</summary>
A Langflow API key is a user-specific token you can use with Langflow.
It is **only** used for sending requests to the Langflow server.
It does **not** access to OpenRAG.
To create a Langflow API key, do the following:
1. In Langflow, click your user icon, and then select **Settings**.
2. Click **Langflow API Keys**, and then click <Icon name="Plus" aria-hidden="true"/> **Add New**.
3. Name your key, and then click **Create API Key**.
4. Copy the API key and store it securely.
5. To use your Langflow API key in a request, set a `LANGFLOW_API_KEY` environment variable in your terminal, and then include an `x-api-key` header or query parameter with your request.
For example:
```bash
# Set variable
export LANGFLOW_API_KEY="sk..."
# Send request
curl --request POST \
--url "http://LANGFLOW_SERVER_ADDRESS/api/v1/run/FLOW_ID" \
--header "Content-Type: application/json" \
--header "x-api-key: $LANGFLOW_API_KEY" \
--data '{
"output_type": "chat",
"input_type": "chat",
"input_value": "Hello"
}'
```
</details>
## Find your way around ## Find your way around
@ -44,12 +77,14 @@ If you aren't getting the results you need, you can further tune the knowledge i
To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**. To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**.
In this example, you'll try a different LLM to demonstrate how the Agent's response changes. In this example, you'll try a different LLM to demonstrate how the Agent's response changes.
You can only change the **Language model**, and not the **Model provider** that you started with in OpenRAG.
If you're using Ollama, you can swap out any model.
1. To edit the Agent's behavior, click **Edit in Langflow**. 1. To edit the Agent's behavior, click **Edit in Langflow**.
You can more quickly access the **Language Model** and **Agent Instructions** fields in this page, but for illustration purposes, navigate to the Langflow visual builder.
2. OpenRAG warns you that you're entering Langflow. Click **Proceed**. 2. OpenRAG warns you that you're entering Langflow. Click **Proceed**.
3. The OpenRAG Open Search Agent flow appears. 3. The OpenRAG Open Search Agent flow appears.
![OpenRAG Open Search Agent Flow](/img/opensearch-agent-flow.png)
![OpenRAG Open Search Agent Flow](/img/opensearch-agent-flow.png)
4. In the **Language Model** component, under **Model Provider**, select **Anthropic**. 4. In the **Language Model** component, under **Model Provider**, select **Anthropic**.
:::note :::note