more-on-local-models

This commit is contained in:
Mendon Kissling 2025-09-26 17:11:14 -04:00
parent bc9f181abd
commit 18a41f2ead
2 changed files with 41 additions and 5 deletions

View file

@ -83,11 +83,12 @@ For more information on virtual environments, see [uv](https://docs.astral.sh/uv
The application opens.
8. Select your language model and embedding model provider, and complete the required fields.
**Your provider can only be selected once, and you must use the same provider for your language model and embedding model.
To change your selection, you must restart OpenRAG.**
To change your provider selection, you must restart OpenRAG and delete the `config.yml` file.**
**Your language model can be changed, but your embeddings model cannot be changed.**
<Tabs groupId="Embedding provider">
<TabItem value="OpenAI" label="OpenAI" default>
9. You already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, so enable **Get API key from environment variable**.
9. If you already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, enable **Get API key from environment variable**.
10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**.
11. To load 2 sample PDFs, enable **Sample dataset**.
This is recommended, but not required.

View file

@ -12,7 +12,40 @@ Get started with OpenRAG by loading your knowledge, swapping out your language m
## Prerequisites
- [Install and start OpenRAG](/install)
- [Langflow API key](/)
- Create a [Langflow API key](https://docs.langflow.org/api-keys-and-authentication)
<details>
<summary>Create a Langflow API key</summary>
A Langflow API key is a user-specific token you can use with Langflow.
It is **only** used for sending requests to the Langflow server.
It does **not** access to OpenRAG.
To create a Langflow API key, do the following:
1. In Langflow, click your user icon, and then select **Settings**.
2. Click **Langflow API Keys**, and then click <Icon name="Plus" aria-hidden="true"/> **Add New**.
3. Name your key, and then click **Create API Key**.
4. Copy the API key and store it securely.
5. To use your Langflow API key in a request, set a `LANGFLOW_API_KEY` environment variable in your terminal, and then include an `x-api-key` header or query parameter with your request.
For example:
```bash
# Set variable
export LANGFLOW_API_KEY="sk..."
# Send request
curl --request POST \
--url "http://LANGFLOW_SERVER_ADDRESS/api/v1/run/FLOW_ID" \
--header "Content-Type: application/json" \
--header "x-api-key: $LANGFLOW_API_KEY" \
--data '{
"output_type": "chat",
"input_type": "chat",
"input_value": "Hello"
}'
```
</details>
## Find your way around
@ -44,12 +77,14 @@ If you aren't getting the results you need, you can further tune the knowledge i
To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**.
In this example, you'll try a different LLM to demonstrate how the Agent's response changes.
You can only change the **Language model**, and not the **Model provider** that you started with in OpenRAG.
If you're using Ollama, you can swap out any model.
1. To edit the Agent's behavior, click **Edit in Langflow**.
You can more quickly access the **Language Model** and **Agent Instructions** fields in this page, but for illustration purposes, navigate to the Langflow visual builder.
2. OpenRAG warns you that you're entering Langflow. Click **Proceed**.
3. The OpenRAG Open Search Agent flow appears.
![OpenRAG Open Search Agent Flow](/img/opensearch-agent-flow.png)
![OpenRAG Open Search Agent Flow](/img/opensearch-agent-flow.png)
4. In the **Language Model** component, under **Model Provider**, select **Anthropic**.
:::note