alekszievr
4f2745504c
Calculate official hotpot EM and F1 scores ( #292 )
2024-12-10 19:16:12 +01:00
Boris
348610e73c
fix: refactor get_graph_from_model to return nodes and edges correctly ( #257 )
...
* fix: handle rate limit error coming from llm model
* fix: fixes lost edges and nodes in get_graph_from_model
* fix: fixes database pruning issue in pgvector (#261 )
* fix: cognee_demo notebook pipeline is not saving summaries
---------
Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
2024-12-06 12:52:01 +01:00
Boris Arzentar
d49ab4c3b5
feat: update code-graph notebook
2024-12-03 23:48:12 +01:00
Boris Arzentar
b89a4b8054
Merge remote-tracking branch 'origin/main' into code-graph
2024-12-03 21:14:19 +01:00
Rita Aleksziev
a0d5102bd8
add some spaces for readability
2024-12-03 17:22:23 +01:00
Rita Aleksziev
0fbb50960b
prompt renaming
2024-12-03 15:59:03 +01:00
Rita Aleksziev
dc082de4c2
minor bugfix in folder creation
2024-12-02 14:54:40 +01:00
Rita Aleksziev
f966f099fc
Prompt renaming to more specific names. Minor code changes.
2024-12-02 12:18:00 +01:00
Boris Arzentar
11acabdb6a
fix: remove duplicate nodes and edges before saving; Fix FalkorDB vector index;
2024-12-02 10:10:18 +01:00
Rita Aleksziev
a4c56f118d
Connect code graph pipeline + retriever + benchmarking
2024-11-29 15:24:49 +01:00
Rita Aleksziev
4da1657140
merge changes from code-graph
2024-11-29 12:16:36 +01:00
Rita Aleksziev
8f241fa6c5
convert edge to string
2024-11-29 12:05:52 +01:00
Leon Luithlen
a5ae9185cd
Replicate PR 33
2024-11-29 11:40:51 +01:00
Leon Luithlen
d9fc740ec0
Fix merge conflicts
2024-11-29 11:33:05 +01:00
Leon Luithlen
b46af5a6f6
Update eval_swe_bench
2024-11-29 11:31:03 +01:00
Leon Luithlen
618d476c30
Add code formating to usermod command
2024-11-29 11:30:39 +01:00
Leon Luithlen
5036f3a85f
Add -y to setup_ubuntu_instance.sh commands and update EC2_README
2024-11-29 11:30:39 +01:00
Leon Luithlen
1bfa3a0ea3
Rebase onto code-graph
2024-11-29 11:30:30 +01:00
Rita Aleksziev
996b3a658b
add custom metric implementation
2024-11-28 16:53:33 +01:00
Rita Aleksziev
8edfe7c5a4
feat/connect code graph pipeline to benchmarking
2024-11-28 16:52:54 +01:00
Boris Arzentar
2408fd7a01
fix: falkordb adapter errors
2024-11-28 09:12:37 +01:00
Rita Aleksziev
4aa634d5e1
Eval function takes eval_metric as input. Works with deepeval metrics like AnswerRelevancyMetric
2024-11-27 16:14:05 +01:00
Rita Aleksziev
f47b185a9e
feat/add correctness score calculation with LLM as a judge
2024-11-27 10:53:48 +01:00
Boris
64b8aac86f
feat: code graph swe integration
...
Co-authored-by: hajdul88 <52442977+hajdul88@users.noreply.github.com>
Co-authored-by: hande-k <handekafkas7@gmail.com>
Co-authored-by: Igor Ilic <igorilic03@gmail.com>
Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
Co-authored-by: Igor Ilic <30923996+dexters1@users.noreply.github.com>
2024-11-27 09:32:29 +01:00
Rita Aleksziev
e1d8f3ea86
use acreate_structured_output instead of create_structured_output in eval script
2024-11-20 16:02:15 +01:00
Rita Aleksziev
2948089806
Read patch generation instructions from file
2024-11-19 14:07:53 +01:00
Rita Aleksziev
838d98238a
Code cleanup
2024-11-19 13:54:04 +01:00
Rita Aleksziev
d986e7c981
minor code cleanup
2024-11-18 15:59:18 +01:00
Rita Aleksziev
98e3445c2c
running swebench evaluation as subprocess
2024-11-18 15:12:36 +01:00
Rita Aleksziev
ed08cdb9f9
using the code graph pipeline instead of cognify
2024-11-15 17:56:19 +01:00
Rita Aleksziev
721fde3d60
generating testspecs for data
2024-11-15 17:14:43 +01:00
Rita Aleksziev
094ba7233e
Running inference with and without cognee
2024-11-14 16:28:03 +01:00
Rita Aleksziev
aa95aa21af
downloading example repo for eval
2024-11-12 17:40:42 +01:00
Rita
26d914eebc
Loading SWE-bench dataset and converting it to DeepEval dataset
2024-11-11 16:48:06 +00:00
Boris
dc187a81d7
feat: migrate search to tasks ( #144 )
...
* fix: don't return anything on health endpoint
* feat: add alembic migrations
* feat: align search types with the data we store and migrate search to tasks
2024-10-07 14:41:35 +02:00
Boris Arzentar
f79631d5da
fix: allow alternative vector db engine to be used
2024-06-06 12:31:55 +02:00
Boris Arzentar
4fb3dc31a4
fix: enable sdk and fix config
2024-06-03 14:03:24 +02:00
Boris Arzentar
aef78c4a8f
fix: set config before using it
2024-05-27 14:18:39 +02:00
Vasilije
9569441c5e
add test for linter
2024-05-25 22:18:07 +02:00
Vasilije
3fadb277cb
Fixes for searches
2024-05-21 19:34:18 +02:00
Vasilije
63356f242a
Fixes to search and evals
2024-05-21 10:03:52 +02:00
Vasilije
8ef23731a3
Updated evals, added falkordb
2024-05-20 14:41:08 +02:00
Vasilije
d099cae128
Add evals for cognee
2024-05-19 22:39:47 +02:00
Vasilije
2657aa7096
Add evals for cognee
2024-05-19 20:35:54 +02:00
