| .github | ||
| assets | ||
| bin | ||
| level_1 | ||
| level_2 | ||
| level_3 | ||
| level_4 | ||
| .gitignore | ||
| Dashboard_example.png | ||
| infographic_final.png | ||
| LICENSE | ||
| README.md | ||
PromethAI-Memory
AI Applications and RAGs - Cognitive Architecture, Testability, Production Ready Apps
Open-source framework for building and testing RAGs and Cognitive Architectures, designed for accuracy, transparency, and control.
Share promethAI Repository
This repo is built to test and evolve RAG architecture, inspired by human cognitive processes, using Python. It's aims to be production ready, testable, but give great visibility in how we build RAG applications.
This project is a part of the PromethAI ecosystem.
It runs in iterations, with each iteration building on the previous one.
Keep Ithaka always in your mind. Arriving there is what you’re destined for. But don’t hurry the journey at all. Better if it lasts for years
Installation
Run the level 4
Make sure you have Docker, Poetry, and Python 3.11 installed and postgres installed.
Copy the .env.example to .env and fill in the variables
poetry shell
docker compose up
Run
python main.py
If you are running natively, change ENVIRONMENT to local in the .env file If you are running in docker, change ENVIRONMENT to postgres in the .env file
Run
python main.py
Or run
``` docker compose up promethai-mem ```
And send API requests add-memory, user-query-to-graph, document-to-graph-db, user-query-processor to the locahost:8000
Current Focus
Cognitive Architecture manager :
- stores data in Vector Database
- uses Graph database to create connection between the terms nad objects
- uses classifiers to pick the right document
- generates context for LLM to process
Project Structure
Level 1 - OpenAI functions + Pydantic + DLTHub
Scope: Give PDFs to the model and get the output in a structured format Blog post: Link We introduce the following concepts:
- Structured output with Pydantic
- CMD script to process custom PDFs
Level 2 - Memory Manager + Metadata management
Scope: Give PDFs to the model and consolidate with the previous user activity and more Blog post: Link We introduce the following concepts:
- Long Term Memory -> store and format the data
- Episodic Buffer -> isolate the working memory
- Attention Modulators -> improve semantic search
- Docker
- API
Level 3 - Dynamic Graph Memory Manager + DB + Rag Test Manager
Scope: Store the data in N-related stores and test the retrieval with the Rag Test Manager Blog post: Link
- Dynamic Memory Manager -> store the data in N hierarchical stores
- Auto-generation of tests
- Multiple file formats supported
- Postgres DB to store metadata
- Docker
- API
- Superset to visualize the results
Level 4 - Dynamic Graph Memory Manager + DB + Rag Test Manager
Scope: Use Neo4j to map the user queries into a knowledge graph based on cognitive architecture Blog post: Soon!
- Dynamic Memory Manager -> store the data in N hierarchical stores
- Dynamic Graph -> map the user queries into a knowledge graph
- Classification -> classify the user queries and choose relevant graph nodes
- Context manager -> generate context for LLM to process containing Semantic, Episodic and Vector store data
- Postgres DB to store metadata
- Docker
- API