gmakstutis/cognee
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: move ec2 setup file and remove extra steps [cog-1585] (#653)

Browse source 
<!-- .github/pull_request_template.md -->

## Description
This .sh file can be used for EC2 deployment as explained in
https://github.com/topoteretes/cognee-docs/pull/58

## DCO Affirmation
I affirm that all code in every commit of this pull request conforms to
the terms of the Topoteretes Developer Certificate of Origin


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Documentation**
- Removed outdated guidance for setting up evaluation environments,
streamlining the visible instructions.

- **Chores**
- Updated the Ubuntu setup process to install Python 3.12, ensuring the
virtual environment uses the latest version and enhancing overall
performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
alekszievr 2025-03-19 15:02:55 +01:00 • committed by GitHub
parent 1c65682242
commit 8dd575e004
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 2 additions and 69 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
evals/cloud/setup_ubuntu_instance.sh → deployment/setup_ubuntu_instance.sh
View file

@ -15,19 +15,16 @@ sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plug
sudo docker run hello-world sudo docker run hello-world
sudo apt install -y unzip
sudo apt-get install -y python3-virtualenv sudo apt-get install -y python3-virtualenv
sudo add-apt-repository -y ppa:deadsnakes/ppa sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt update -y sudo apt update -y
sudo apt install -y python3.11 sudo apt install -y python3.12
virtualenv venv --python=python3.11 virtualenv venv --python=python3.12
source venv/bin/activate source venv/bin/activate
pip install poetry pip install poetry
poetry install poetry install
pip install swebench transformers sentencepiece datasets tiktoken protobuf

64
evals/EC2_README.md
View file

@ -1,64 +0,0 @@
## Creating the EC2 Instance
Create an EC2 Instance with the
`Ubuntu Image`
Many instance types will work, we used:
`m7a.2xlarge` # more than 8 parallel processes doesn't seem to speed up overall process. Maybe to do with docker parallelism?
DON'T FORGET TO ADD
`500 GB storage`
Or the evaluation run will run out of space
Add a key pair login where you have access to the corresponding key file (*.pem)
## Accessing your instance and setup
To ssh into the instance, you have to save your key pair file (*.pem) to an appropriate location, such as ~/.aws. After launching the instance, you can access the Instance Summary, and retrieve "Public IPv4 DNS" address. Then run
`ssh -i PATH_TO_KEY ubuntu@IPv4ADDRESS`
to gain command line access to the instance.
To copy your current state of cognee, go to the folder that contains "cognee" on your local machine, zip it to cognee.zip and run:
`zip -r cognee.zip cognee`
`scp -i PATH_TO_KEY cognee.zip ubuntu@IPv4ADDRESS:cognee.zip`
And unzip cognee.zip in your SSH session:
`sudo apt install unzip`
`unzip cognee.zip`
Then run:
`cd cognee`
`source evals/cloud/setup_ubuntu_instance.sh`
`sudo usermod -aG docker $USER`
disconnect, and reconnect.
Confirm that `ubuntu` has been added to the docker user group with
`groups | grep docker`
## Running SWE-bench
Then enter a `screen` and activate the virtual env
`screen`
`source venv/bin/activate`
then, from cognee, you can run swe_bench:
`cd cognee`
`python evals/eval_swe_bench.py --cognee_off --max_workers=N_CPUS`
Building the environment images should take roughly 17 minutes
If the virtual env wasn't set up correctly for some reason, just run the last few lines of `setup_ubuntu_instance.sh` manually