bb679c2dd7
* Update cognify and the networkx client to prepare for running in Neo4j * Fix for openai model * Add the fix to the infra so that the models can be passed to the library. Enable llm_provider to be passed. * Auto graph generation now works with neo4j * Added fixes for both neo4j and networkx * Explicitly name semantic node connections * Added updated docs, readme, chunkers and updates to cognify * Make docs build trigger only when changes on it happen * Update docs, test git actions * Separate cognify logic into tasks * Introduce dspy knowledge graph extraction --------- Co-authored-by: Boris Arzentar <borisarzentar@gmail.com>
1.3 KiB
1.3 KiB
Running cognee with local models
🚀 Getting Started with Local Models
You'll need to run the local model on your machine or use one of the providers hosting the model. !!! note "We had some success with mixtral, but 7b models did not work well. We recommend using mixtral for now."
Ollama
Set up Ollama by following instructions on Ollama website
Set the environment variable to use the model
LLM_PROVIDER = 'ollama'
Otherwise, you can set the configuration for the model:
from cognee.infrastructure import infrastructure_config
infrastructure_config.set_config({
"llm_provider": 'ollama'
})
You can also set the HOST and model name:
CUSTOM_OLLAMA_ENDPOINT= "http://localhost:11434/v1"
CUSTOM_OLLAMA_MODEL = "mistral:instruct"
Anyscale
LLM_PROVIDER = 'custom'
Otherwise, you can set the configuration for the model:
from cognee.infrastructure import infrastructure_config
infrastructure_config.set_config({
"llm_provider": 'custom'
})
You can also set the HOST and model name:
CUSTOM_LLM_MODEL = "mistralai/Mixtral-8x7B-Instruct-v0.1"
CUSTOM_ENDPOINT = "https://api.endpoints.anyscale.com/v1"
CUSTOM_LLM_API_KEY = "your_api_key"
You can set the same way HOST and model name for any other provider that has an API endpoint.