more-on-local-models
This commit is contained in:
parent
bc9f181abd
commit
18a41f2ead
2 changed files with 41 additions and 5 deletions
|
|
@ -83,11 +83,12 @@ For more information on virtual environments, see [uv](https://docs.astral.sh/uv
|
||||||
The application opens.
|
The application opens.
|
||||||
8. Select your language model and embedding model provider, and complete the required fields.
|
8. Select your language model and embedding model provider, and complete the required fields.
|
||||||
**Your provider can only be selected once, and you must use the same provider for your language model and embedding model.
|
**Your provider can only be selected once, and you must use the same provider for your language model and embedding model.
|
||||||
To change your selection, you must restart OpenRAG.**
|
To change your provider selection, you must restart OpenRAG and delete the `config.yml` file.**
|
||||||
|
**Your language model can be changed, but your embeddings model cannot be changed.**
|
||||||
|
|
||||||
<Tabs groupId="Embedding provider">
|
<Tabs groupId="Embedding provider">
|
||||||
<TabItem value="OpenAI" label="OpenAI" default>
|
<TabItem value="OpenAI" label="OpenAI" default>
|
||||||
9. You already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, so enable **Get API key from environment variable**.
|
9. If you already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, enable **Get API key from environment variable**.
|
||||||
10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**.
|
10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**.
|
||||||
11. To load 2 sample PDFs, enable **Sample dataset**.
|
11. To load 2 sample PDFs, enable **Sample dataset**.
|
||||||
This is recommended, but not required.
|
This is recommended, but not required.
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,40 @@ Get started with OpenRAG by loading your knowledge, swapping out your language m
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- [Install and start OpenRAG](/install)
|
- [Install and start OpenRAG](/install)
|
||||||
- [Langflow API key](/)
|
- Create a [Langflow API key](https://docs.langflow.org/api-keys-and-authentication)
|
||||||
|
<details>
|
||||||
|
<summary>Create a Langflow API key</summary>
|
||||||
|
|
||||||
|
A Langflow API key is a user-specific token you can use with Langflow.
|
||||||
|
It is **only** used for sending requests to the Langflow server.
|
||||||
|
It does **not** access to OpenRAG.
|
||||||
|
|
||||||
|
To create a Langflow API key, do the following:
|
||||||
|
|
||||||
|
1. In Langflow, click your user icon, and then select **Settings**.
|
||||||
|
2. Click **Langflow API Keys**, and then click <Icon name="Plus" aria-hidden="true"/> **Add New**.
|
||||||
|
3. Name your key, and then click **Create API Key**.
|
||||||
|
4. Copy the API key and store it securely.
|
||||||
|
5. To use your Langflow API key in a request, set a `LANGFLOW_API_KEY` environment variable in your terminal, and then include an `x-api-key` header or query parameter with your request.
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set variable
|
||||||
|
export LANGFLOW_API_KEY="sk..."
|
||||||
|
|
||||||
|
# Send request
|
||||||
|
curl --request POST \
|
||||||
|
--url "http://LANGFLOW_SERVER_ADDRESS/api/v1/run/FLOW_ID" \
|
||||||
|
--header "Content-Type: application/json" \
|
||||||
|
--header "x-api-key: $LANGFLOW_API_KEY" \
|
||||||
|
--data '{
|
||||||
|
"output_type": "chat",
|
||||||
|
"input_type": "chat",
|
||||||
|
"input_value": "Hello"
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
## Find your way around
|
## Find your way around
|
||||||
|
|
||||||
|
|
@ -44,12 +77,14 @@ If you aren't getting the results you need, you can further tune the knowledge i
|
||||||
To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**.
|
To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**.
|
||||||
|
|
||||||
In this example, you'll try a different LLM to demonstrate how the Agent's response changes.
|
In this example, you'll try a different LLM to demonstrate how the Agent's response changes.
|
||||||
|
You can only change the **Language model**, and not the **Model provider** that you started with in OpenRAG.
|
||||||
|
If you're using Ollama, you can swap out any model.
|
||||||
|
|
||||||
1. To edit the Agent's behavior, click **Edit in Langflow**.
|
1. To edit the Agent's behavior, click **Edit in Langflow**.
|
||||||
|
You can more quickly access the **Language Model** and **Agent Instructions** fields in this page, but for illustration purposes, navigate to the Langflow visual builder.
|
||||||
2. OpenRAG warns you that you're entering Langflow. Click **Proceed**.
|
2. OpenRAG warns you that you're entering Langflow. Click **Proceed**.
|
||||||
3. The OpenRAG Open Search Agent flow appears.
|
3. The OpenRAG Open Search Agent flow appears.
|
||||||
|

|
||||||

|
|
||||||
|
|
||||||
4. In the **Language Model** component, under **Model Provider**, select **Anthropic**.
|
4. In the **Language Model** component, under **Model Provider**, select **Anthropic**.
|
||||||
:::note
|
:::note
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue