more-on-local-models
This commit is contained in:
parent
bc9f181abd
commit
18a41f2ead
2 changed files with 41 additions and 5 deletions
|
|
@ -83,11 +83,12 @@ For more information on virtual environments, see [uv](https://docs.astral.sh/uv
|
|||
The application opens.
|
||||
8. Select your language model and embedding model provider, and complete the required fields.
|
||||
**Your provider can only be selected once, and you must use the same provider for your language model and embedding model.
|
||||
To change your selection, you must restart OpenRAG.**
|
||||
To change your provider selection, you must restart OpenRAG and delete the `config.yml` file.**
|
||||
**Your language model can be changed, but your embeddings model cannot be changed.**
|
||||
|
||||
<Tabs groupId="Embedding provider">
|
||||
<TabItem value="OpenAI" label="OpenAI" default>
|
||||
9. You already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, so enable **Get API key from environment variable**.
|
||||
9. If you already entered a value for `OPENAI_API_KEY` in the TUI in Step 5, enable **Get API key from environment variable**.
|
||||
10. Under **Advanced settings**, select your **Embedding Model** and **Language Model**.
|
||||
11. To load 2 sample PDFs, enable **Sample dataset**.
|
||||
This is recommended, but not required.
|
||||
|
|
|
|||
|
|
@ -12,7 +12,40 @@ Get started with OpenRAG by loading your knowledge, swapping out your language m
|
|||
## Prerequisites
|
||||
|
||||
- [Install and start OpenRAG](/install)
|
||||
- [Langflow API key](/)
|
||||
- Create a [Langflow API key](https://docs.langflow.org/api-keys-and-authentication)
|
||||
<details>
|
||||
<summary>Create a Langflow API key</summary>
|
||||
|
||||
A Langflow API key is a user-specific token you can use with Langflow.
|
||||
It is **only** used for sending requests to the Langflow server.
|
||||
It does **not** access to OpenRAG.
|
||||
|
||||
To create a Langflow API key, do the following:
|
||||
|
||||
1. In Langflow, click your user icon, and then select **Settings**.
|
||||
2. Click **Langflow API Keys**, and then click <Icon name="Plus" aria-hidden="true"/> **Add New**.
|
||||
3. Name your key, and then click **Create API Key**.
|
||||
4. Copy the API key and store it securely.
|
||||
5. To use your Langflow API key in a request, set a `LANGFLOW_API_KEY` environment variable in your terminal, and then include an `x-api-key` header or query parameter with your request.
|
||||
For example:
|
||||
|
||||
```bash
|
||||
# Set variable
|
||||
export LANGFLOW_API_KEY="sk..."
|
||||
|
||||
# Send request
|
||||
curl --request POST \
|
||||
--url "http://LANGFLOW_SERVER_ADDRESS/api/v1/run/FLOW_ID" \
|
||||
--header "Content-Type: application/json" \
|
||||
--header "x-api-key: $LANGFLOW_API_KEY" \
|
||||
--data '{
|
||||
"output_type": "chat",
|
||||
"input_type": "chat",
|
||||
"input_value": "Hello"
|
||||
}'
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Find your way around
|
||||
|
||||
|
|
@ -44,12 +77,14 @@ If you aren't getting the results you need, you can further tune the knowledge i
|
|||
To modify the knowledge ingestion or Agent behavior, click <Icon name="Settings" aria-hidden="true"/> **Settings**.
|
||||
|
||||
In this example, you'll try a different LLM to demonstrate how the Agent's response changes.
|
||||
You can only change the **Language model**, and not the **Model provider** that you started with in OpenRAG.
|
||||
If you're using Ollama, you can swap out any model.
|
||||
|
||||
1. To edit the Agent's behavior, click **Edit in Langflow**.
|
||||
You can more quickly access the **Language Model** and **Agent Instructions** fields in this page, but for illustration purposes, navigate to the Langflow visual builder.
|
||||
2. OpenRAG warns you that you're entering Langflow. Click **Proceed**.
|
||||
3. The OpenRAG Open Search Agent flow appears.
|
||||
|
||||

|
||||

|
||||
|
||||
4. In the **Language Model** component, under **Model Provider**, select **Anthropic**.
|
||||
:::note
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue