gmakstutis/cognee
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

edit cognee_hotpot_eval.ipynb

Browse source
This commit is contained in:
hande-k 2025-01-24 13:20:38 +01:00
parent bc1b05437e
commit 52e5b5c6f4
1 changed files with 13 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
notebooks/cognee_hotpot_eval.ipynb
View file

@ -56,8 +56,8 @@
"source": [ "source": [
"## Define Context Provider\n", "## Define Context Provider\n",
"**Options**: \n", "**Options**: \n",
"- **no_rag**: raw context \n",
"- **cognee**: context with cognee \n", "- **cognee**: context with cognee \n",
"- **no_rag**: raw context \n",
"- **simple_rag**: context with simple rag \n", "- **simple_rag**: context with simple rag \n",
"- **brute_force**: context with brute force triplet search" "- **brute_force**: context with brute force triplet search"
] ]
@ -66,7 +66,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### Here, \"no_rag\" is used as context provider\n" "### Here, \"cognee\" is used as context provider\n"
] ]
}, },
{ {
@ -75,7 +75,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"context_provider_name = \"no_rag\"\n", "context_provider_name = \"cognee\"\n",
"context_provider = qa_context_providers[context_provider_name]" "context_provider = qa_context_providers[context_provider_name]"
] ]
}, },
@ -282,7 +282,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"F1\"`" "##### Calculating `\"F1 Score\"`"
] ]
}, },
{ {
@ -320,7 +320,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"EM\"`" "##### Calculating `\"Exact Match (EM) Score\"`"
] ]
}, },
{ {
@ -350,7 +350,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### \"cognee\" as context provider\n" "### \"no_rag\" as context provider"
] ]
}, },
{ {
@ -359,7 +359,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"context_provider_name = \"cognee\"\n", "context_provider_name = \"no_rag\"\n",
"context_provider = qa_context_providers[context_provider_name]" "context_provider = qa_context_providers[context_provider_name]"
] ]
}, },
@ -553,7 +553,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"F1\"`" "##### Calculating `\"F1 Score\"`"
] ]
}, },
{ {
@ -591,7 +591,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"EM\"`" "##### Calculating `\"Exact Match (EM) Score\"`"
] ]
}, },
{ {
@ -833,7 +833,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"metric_name_list = [\"F1\"]\n", "metric_name_list = [\"F1 Score\"]\n",
"eval_metrics = get_metrics(metric_name_list)" "eval_metrics = get_metrics(metric_name_list)"
] ]
}, },
@ -862,7 +862,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"EM\"`" "##### Calculating `\"Exact Match (EM) Score\"`"
] ]
}, },
{ {
@ -1095,7 +1095,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"F1\"`" "##### Calculating `\"F1 Score\"`"
] ]
}, },
{ {
@ -1133,7 +1133,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"##### Calculating `\"EM\"`" "##### Calculating `\"Exact Match (EM) Score\"`"
] ]
}, },
{ {