1353 lines
93 KiB
Text
1353 lines
93 KiB
Text
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "13be50c7-167c-4a03-bd75-53904baa1f8c",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "8a8942b5-91d6-4746-b35d-00f58bc16d7b",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import logging\n",
|
||
"\n",
|
||
"from langchain.prompts import ChatPromptTemplate\n",
|
||
"import json\n",
|
||
"from langchain.document_loaders import TextLoader\n",
|
||
"from langchain.document_loaders import DirectoryLoader\n",
|
||
"from langchain.chains import create_extraction_chain\n",
|
||
"from langchain.chat_models import ChatOpenAI\n",
|
||
"import re\n",
|
||
"\n",
|
||
"from dotenv import load_dotenv\n",
|
||
"import os\n",
|
||
"\n",
|
||
"# Load environment variables from .env file\n",
|
||
"load_dotenv()\n",
|
||
"import instructor\n",
|
||
"from openai import OpenAI\n",
|
||
"\n",
|
||
"\n",
|
||
"aclient = instructor.patch(OpenAI())\n",
|
||
"\n",
|
||
"from typing import Optional, List\n",
|
||
"from pydantic import BaseModel, Field\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "14484e25-fae8-4306-b03f-dae91fe5d0aa",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"input = \"\"\" The economist J.K. Galbraith once wrote, “Faced with a choice between changing one’s mind and proving there is no need to do so, almost everyone gets busy with the proof.”\n",
|
||
"\n",
|
||
"Leo Tolstoy was even bolder: “The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him.”\n",
|
||
"\n",
|
||
"What’s going on here? Why don’t facts change our minds? And why would someone continue to believe a false or inaccurate idea anyway? How do such behaviors serve us?\n",
|
||
"The Logic of False Beliefs\n",
|
||
"Humans need a reasonably accurate view of the world in order to survive. If your model of reality is wildly different from the actual world, then you struggle to take effective actions each day.\n",
|
||
"\n",
|
||
"However, truth and accuracy are not the only things that matter to the human mind. Humans also seem to have a deep desire to belong.\n",
|
||
"\n",
|
||
"In Atomic Habits, I wrote, “Humans are herd animals. We want to fit in, to bond with others, and to earn the respect and approval of our peers. Such inclinations are essential to our survival. For most of our evolutionary history, our ancestors lived in tribes. Becoming separated from the tribe—or worse, being cast out—was a death sentence.”\n",
|
||
"\n",
|
||
"Understanding the truth of a situation is important, but so is remaining part of a tribe. While these two desires often work well together, they occasionally come into conflict.\n",
|
||
"\n",
|
||
"In many circumstances, social connection is actually more helpful to your daily life than understanding the truth of a particular fact or idea. The Harvard psychologist Steven Pinker put it this way, “People are embraced or condemned according to their beliefs, so one function of the mind may be to hold beliefs that bring the belief-holder the greatest number of allies, protectors, or disciples, rather than beliefs that are most likely to be true.”\n",
|
||
"\n",
|
||
"We don’t always believe things because they are correct. Sometimes we believe things because they make us look good to the people we care about.\n",
|
||
"\n",
|
||
"I thought Kevin Simler put it well when he wrote, “If a brain anticipates that it will be rewarded for adopting a particular belief, it’s perfectly happy to do so, and doesn’t much care where the reward comes from — whether it’s pragmatic (better outcomes resulting from better decisions), social (better treatment from one’s peers), or some mix of the two.”\n",
|
||
"\n",
|
||
"False beliefs can be useful in a social sense even if they are not useful in a factual sense. For lack of a better phrase, we might call this approach “factually false, but socially accurate.” When we have to choose between the two, people often select friends and family over facts.\n",
|
||
"\n",
|
||
"This insight not only explains why we might hold our tongue at a dinner party or look the other way when our parents say something offensive, but also reveals a better way to change the minds of others.\n",
|
||
"\n",
|
||
"Facts Don’t Change Our Minds. Friendship Does.\n",
|
||
"Convincing someone to change their mind is really the process of convincing them to change their tribe. If they abandon their beliefs, they run the risk of losing social ties. You can’t expect someone to change their mind if you take away their community too. You have to give them somewhere to go. Nobody wants their worldview torn apart if loneliness is the outcome.\n",
|
||
"\n",
|
||
"The way to change people’s minds is to become friends with them, to integrate them into your tribe, to bring them into your circle. Now, they can change their beliefs without the risk of being abandoned socially.\n",
|
||
"\n",
|
||
"The British philosopher Alain de Botton suggests that we simply share meals with those who disagree with us:\n",
|
||
"\n",
|
||
"“Sitting down at a table with a group of strangers has the incomparable and odd benefit of making it a little more difficult to hate them with impunity. Prejudice and ethnic strife feed off abstraction. However, the proximity required by a meal – something about handing dishes around, unfurling napkins at the same moment, even asking a stranger to pass the salt – disrupts our ability to cling to the belief that the outsiders who wear unusual clothes and speak in distinctive accents deserve to be sent home or assaulted. For all the large-scale political solutions which have been proposed to salve ethnic conflict, there are few more effective ways to promote tolerance between suspicious neighbours than to force them to eat supper together.”\n",
|
||
"\n",
|
||
"Perhaps it is not difference, but distance that breeds tribalism and hostility. As proximity increases, so does understanding. I am reminded of Abraham Lincoln’s quote, “I don’t like that man. I must get to know him better.”\n",
|
||
"\n",
|
||
"Facts don’t change our minds. Friendship does.\n",
|
||
"\n",
|
||
"The Spectrum of Beliefs\n",
|
||
"Years ago, Ben Casnocha mentioned an idea to me that I haven’t been able to shake: The people who are most likely to change our minds are the ones we agree with on 98 percent of topics.\n",
|
||
"\n",
|
||
"If someone you know, like, and trust believes a radical idea, you are more likely to give it merit, weight, or consideration. You already agree with them in most areas of life. Maybe you should change your mind on this one too. But if someone wildly different than you proposes the same radical idea, well, it’s easy to dismiss them as a crackpot.\n",
|
||
"\n",
|
||
"One way to visualize this distinction is by mapping beliefs on a spectrum. If you divide this spectrum into 10 units and you find yourself at Position 7, then there is little sense in trying to convince someone at Position 1. The gap is too wide. When you’re at Position 7, your time is better spent connecting with people who are at Positions 6 and 8, gradually pulling them in your direction.\n",
|
||
"\n",
|
||
"The most heated arguments often occur between people on opposite ends of the spectrum, but the most frequent learning occurs from people who are nearby. The closer you are to someone, the more likely it becomes that the one or two beliefs you don’t share will bleed over into your own mind and shape your thinking. The further away an idea is from your current position, the more likely you are to reject it outright.\n",
|
||
"\n",
|
||
"When it comes to changing people’s minds, it is very difficult to jump from one side to another. You can’t jump down the spectrum. You have to slide down it.\n",
|
||
"\n",
|
||
"Any idea that is sufficiently different from your current worldview will feel threatening. And the best place to ponder a threatening idea is in a non-threatening environment. As a result, books are often a better vehicle for transforming beliefs than conversations or debates.\n",
|
||
"\n",
|
||
"In conversation, people have to carefully consider their status and appearance. They want to save face and avoid looking stupid. When confronted with an uncomfortable set of facts, the tendency is often to double down on their current position rather than publicly admit to being wrong.\n",
|
||
"\n",
|
||
"Books resolve this tension. With a book, the conversation takes place inside someone’s head and without the risk of being judged by others. It’s easier to be open-minded when you aren’t feeling defensive.\n",
|
||
"\n",
|
||
"Arguments are like a full frontal attack on a person’s identity. Reading a book is like slipping the seed of an idea into a person’s brain and letting it grow on their own terms. There’s enough wrestling going on in someone’s head when they are overcoming a pre-existing belief. They don’t need to wrestle with you too.\n",
|
||
"\n",
|
||
"Why False Ideas Persist\n",
|
||
"There is another reason bad ideas continue to live on, which is that people continue to talk about them.\n",
|
||
"\n",
|
||
"Silence is death for any idea. An idea that is never spoken or written down dies with the person who conceived it. Ideas can only be remembered when they are repeated. They can only be believed when they are repeated.\n",
|
||
"\n",
|
||
"I have already pointed out that people repeat ideas to signal they are part of the same social group. But here’s a crucial point most people miss:\n",
|
||
"\n",
|
||
"People also repeat bad ideas when they complain about them. Before you can criticize an idea, you have to reference that idea. You end up repeating the ideas you’re hoping people will forget—but, of course, people can’t forget them because you keep talking about them. The more you repeat a bad idea, the more likely people are to believe it.\n",
|
||
"\n",
|
||
"Let’s call this phenomenon Clear’s Law of Recurrence: The number of people who believe an idea is directly proportional to the number of times it has been repeated during the last year—even if the idea is false.\n",
|
||
"\n",
|
||
"Each time you attack a bad idea, you are feeding the very monster you are trying to destroy. As one Twitter employee wrote, “Every time you retweet or quote tweet someone you’re angry with, it helps them. It disseminates their BS. Hell for the ideas you deplore is silence. Have the discipline to give it to them.”\n",
|
||
"\n",
|
||
"Your time is better spent championing good ideas than tearing down bad ones. Don’t waste time explaining why bad ideas are bad. You are simply fanning the flame of ignorance and stupidity.\n",
|
||
"\n",
|
||
"The best thing that can happen to a bad idea is that it is forgotten. The best thing that can happen to a good idea is that it is shared. It makes me think of Tyler Cowen’s quote, “Spend as little time as possible talking about how other people are wrong.”\n",
|
||
"\n",
|
||
"Feed the good ideas and let bad ideas die of starvation.\n",
|
||
"\n",
|
||
"\n",
|
||
"\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "e328a903-d084-4d07-9b95-0a9196d7f719",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"\"\"\" We classify input based on the available document types\"\"\"\n",
|
||
"\n",
|
||
"classification = {\n",
|
||
" \"Natural Language Text\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Articles, essays, and reports\",\n",
|
||
" \"Books and manuscripts\",\n",
|
||
" \"News stories and blog posts\",\n",
|
||
" \"Research papers and academic publications\",\n",
|
||
" \"Social media posts and comments\",\n",
|
||
" \"Website content and product descriptions\",\n",
|
||
" \"Personal narratives and stories\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Structured Documents\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Spreadsheets and tables\",\n",
|
||
" \"Forms and surveys\",\n",
|
||
" \"Databases and CSV files\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Code and Scripts\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Source code in various programming languages\",\n",
|
||
" \"Shell commands and scripts\",\n",
|
||
" \"Markup languages (HTML, XML)\",\n",
|
||
" \"Stylesheets (CSS) and configuration files (YAML, JSON, INI)\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Conversational Data\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Chat transcripts and messaging history\",\n",
|
||
" \"Customer service logs and interactions\",\n",
|
||
" \"Conversational AI training data\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Educational Content\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Textbook content and lecture notes\",\n",
|
||
" \"Exam questions and academic exercises\",\n",
|
||
" \"E-learning course materials\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Creative Writing\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Poetry and prose\",\n",
|
||
" \"Scripts for plays, movies, and television\",\n",
|
||
" \"Song lyrics\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Technical Documentation\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Manuals and user guides\",\n",
|
||
" \"Technical specifications and API documentation\",\n",
|
||
" \"Helpdesk articles and FAQs\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Legal and Regulatory Documents\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Contracts and agreements\",\n",
|
||
" \"Laws, regulations, and legal case documents\",\n",
|
||
" \"Policy documents and compliance materials\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Medical and Scientific Texts\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Clinical trial reports\",\n",
|
||
" \"Patient records and case notes\",\n",
|
||
" \"Scientific journal articles\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Financial and Business Documents\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Financial reports and statements\",\n",
|
||
" \"Business plans and proposals\",\n",
|
||
" \"Market research and analysis reports\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Advertising and Marketing Materials\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Ad copies and marketing slogans\",\n",
|
||
" \"Product catalogs and brochures\",\n",
|
||
" \"Press releases and promotional content\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Emails and Correspondence\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Professional and formal correspondence\",\n",
|
||
" \"Personal emails and letters\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Metadata and Annotations\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Image and video captions\",\n",
|
||
" \"Annotations and metadata for various media\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Language Learning Materials\": {\n",
|
||
" \"type\": \"TEXT\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Vocabulary lists and grammar rules\",\n",
|
||
" \"Language exercises and quizzes\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Audio Content\": {\n",
|
||
" \"type\": \"AUDIO\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Music tracks and albums\",\n",
|
||
" \"Podcasts and radio broadcasts\",\n",
|
||
" \"Audiobooks and audio guides\",\n",
|
||
" \"Recorded interviews and speeches\",\n",
|
||
" \"Sound effects and ambient sounds\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Image Content\": {\n",
|
||
" \"type\": \"IMAGE\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Photographs and digital images\",\n",
|
||
" \"Illustrations, diagrams, and charts\",\n",
|
||
" \"Infographics and visual data representations\",\n",
|
||
" \"Artwork and paintings\",\n",
|
||
" \"Screenshots and graphical user interfaces\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Video Content\": {\n",
|
||
" \"type\": \"VIDEO\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Movies and short films\",\n",
|
||
" \"Documentaries and educational videos\",\n",
|
||
" \"Video tutorials and how-to guides\",\n",
|
||
" \"Animated features and cartoons\",\n",
|
||
" \"Live event recordings and sports broadcasts\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Multimedia Content\": {\n",
|
||
" \"type\": \"MULTIMEDIA\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Interactive web content and games\",\n",
|
||
" \"Virtual reality (VR) and augmented reality (AR) experiences\",\n",
|
||
" \"Mixed media presentations and slide decks\",\n",
|
||
" \"E-learning modules with integrated multimedia\",\n",
|
||
" \"Digital exhibitions and virtual tours\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"3D Models and CAD Content\": {\n",
|
||
" \"type\": \"3D_MODEL\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Architectural renderings and building plans\",\n",
|
||
" \"Product design models and prototypes\",\n",
|
||
" \"3D animations and character models\",\n",
|
||
" \"Scientific simulations and visualizations\",\n",
|
||
" \"Virtual objects for AR/VR environments\"\n",
|
||
" ]\n",
|
||
" },\n",
|
||
" \"Procedural Content\": {\n",
|
||
" \"type\": \"PROCEDURAL\",\n",
|
||
" \"subclass\": [\n",
|
||
" \"Tutorials and step-by-step guides\",\n",
|
||
" \"Workflow and process descriptions\",\n",
|
||
" \"Simulation and training exercises\",\n",
|
||
" \"Recipes and crafting instructions\"\n",
|
||
" ]\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"system_prompt = f\"\"\" Classify content based on the following categories: {str(classification)}\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "af1b6a25-b37d-4861-82ae-fd74a7c69bc8",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class CognitiveLayerSubgroup(BaseModel):\n",
|
||
" \"\"\" CognitiveCategorySubgroup in a general category \"\"\"\n",
|
||
" id: int\n",
|
||
" name:str\n",
|
||
" data_type:str\n",
|
||
"\n",
|
||
"\n",
|
||
"class CognitiveCategory(BaseModel):\n",
|
||
" \"\"\"Cognitive category\"\"\"\n",
|
||
" name:str\n",
|
||
" cognitive_subgroups: List[CognitiveLayerSubgroup] = Field(..., default_factory=list)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "cf1965e3-e870-49a7-8ae5-fa4371e1c8f5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def classify_input(input) -> CognitiveCategory:\n",
|
||
" \"\"\"Classify input\"\"\"\n",
|
||
" model = \"gpt-4-1106-preview\"\n",
|
||
" user_prompt = f\"Use the given format to extract information from the following input: {input}.\"\n",
|
||
"\n",
|
||
"\n",
|
||
" out = aclient.chat.completions.create(\n",
|
||
" model=model,\n",
|
||
" messages=[\n",
|
||
" {\n",
|
||
" \"role\": \"user\",\n",
|
||
" \"content\": user_prompt,\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"role\": \"system\",\n",
|
||
" \"content\": system_prompt,\n",
|
||
" },\n",
|
||
" \n",
|
||
" {\n",
|
||
" \"role\": \"system\",\n",
|
||
" \"content\": \"Make sure both values are returned. Incomplete results will result in termination\",\n",
|
||
" },\n",
|
||
" ],\n",
|
||
" response_model=CognitiveCategory,\n",
|
||
" )\n",
|
||
" return out"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "fad0c4b0-cd61-4c3c-9964-47f019278060",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{\"name\":\"Natural Language Text\",\"cognitive_subgroups\":[{\"id\":1,\"name\":\"Articles, essays, and reports\",\"data_type\":\"TEXT\"}]}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"required_layers = classify_input(input = input)\n",
|
||
"print(required_layers.json())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "06b483bf-2fa0-414f-8253-27ffe9a2881c",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{'name': 'Natural Language Text', 'cognitive_subgroups': [{'id': 1, 'name': 'Articles, essays, and reports', 'data_type': 'TEXT'}]}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(required_layers.dict())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "35461aff-fd80-4eb2-94b2-66c742db8e55",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Hello\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(\"Hello\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "57e227bc-363d-437f-9430-c5d14aff6a31",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"TEXT\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(required_layers.dict()['cognitive_subgroups'][0]['data_type'])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "4112063c-e94c-4876-965e-1785e0682329",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"\n",
|
||
"\n",
|
||
"system_prompt = f\"\"\"\n",
|
||
"You are tasked with analyzing a {required_layers.dict()['cognitive_subgroups'][0]['data_type']} files, especially in a multilayer network context for tasks such as analysis, categorization, and feature extraction, various layers can be incorporated to capture the depth and breadth of information contained within the {required_layers.dict()['cognitive_subgroups'][0]['data_type']} \n",
|
||
"These layers can help in understanding the content, context, and characteristics of the {required_layers.dict()['cognitive_subgroups'][0]['data_type']}\n",
|
||
"Your objective is to extract meaningful layers of information that will contribute to constructing a detailed multilayer network or knowledge graph.\n",
|
||
"Approach this task by considering the unique characteristics and inherent properties of the data at hand.\n",
|
||
"VERY IMPORTANT: The context you are working in is {required_layers.dict()['name']} and specific domain you are extracting data on is {required_layers.dict()['cognitive_subgroups'][0]['name']}\n",
|
||
"\n",
|
||
"Guidelines for Layer Extraction:\n",
|
||
"\n",
|
||
"Take into account: The content type that in this case is: {required_layers.dict()['cognitive_subgroups'][0]['name']} should play a major role in how you decompose into layers.\n",
|
||
"\n",
|
||
"Based on your analysis, define and describe the layers you've identified, explaining their relevance and contribution to understanding the dataset. Your independent identification of layers will enable a nuanced and multifaceted representation of the data, enhancing applications in knowledge discovery, content analysis, and information retrieval.\n",
|
||
"\n",
|
||
".\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "7c5baaed-5447-425b-bc9c-03d071d66187",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class CognitiveLayerSubgroup(BaseModel):\n",
|
||
" \"\"\" CognitiveLayerSubgroup in a general layer \"\"\"\n",
|
||
" id: int\n",
|
||
" name:str\n",
|
||
" description: str\n",
|
||
"\n",
|
||
"\n",
|
||
"class CognitiveLayer(BaseModel):\n",
|
||
" \"\"\"Cognitive layer\"\"\"\n",
|
||
" category_name:str\n",
|
||
" cognitive_layers: List[CognitiveLayerSubgroup] = Field(..., default_factory=list)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "ee9c8dad-00ee-48ec-bcb5-8b9c74f91141",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def determine_layers(input) -> CognitiveLayer:\n",
|
||
" \"\"\"Classify input\"\"\"\n",
|
||
" model = \"gpt-4-1106-preview\"\n",
|
||
" user_prompt = f\"Use the given format to extract information from the following input: {input}.\"\n",
|
||
"\n",
|
||
"\n",
|
||
" out = aclient.chat.completions.create(\n",
|
||
" model=model,\n",
|
||
" messages=[\n",
|
||
" {\n",
|
||
" \"role\": \"user\",\n",
|
||
" \"content\": user_prompt,\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"role\": \"system\",\n",
|
||
" \"content\": system_prompt,\n",
|
||
" },\n",
|
||
" ],\n",
|
||
" response_model=CognitiveLayer,\n",
|
||
" )\n",
|
||
" return out"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "f4f59ef6-5cdd-478c-a96e-d2de2cc0e04f",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"cognitive_layers = determine_layers(input=input)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "15631e68-61dc-4955-853f-52bf0cb93fbb",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"category_name='Natural Language Text' cognitive_layers=[CognitiveLayerSubgroup(id=1, name='Thematic Content Layer', description='This layer captures the central themes, topics, or subject matter of the articles, essays, and reports. It involves analyzing the text for key messages, main arguments, and overarching ideologies, which aids in understanding the focus and direction of the content.'), CognitiveLayerSubgroup(id=2, name='Structural Composition Layer', description=\"This layer encompasses the organization and structure of the text, including headings, subheadings, paragraphs, and overall layout. Understanding how the text is organized can reveal the author's approach to presenting information and building their argument.\"), CognitiveLayerSubgroup(id=3, name='Linguistic Style Layer', description=\"This layer analyzes the linguistic choices and writing styles used in the text, such as vocabulary, sentence construction, and use of literary devices. It provides insights into the author's tone, level of formality, and persuasive techniques.\"), CognitiveLayerSubgroup(id=4, name='Contextual Relevance Layer', description='This layer examines the text in relation to its historical, cultural, or situational context. It considers external factors that may influence the interpretation of the text, including the time period it was written in and its cultural significance.'), CognitiveLayerSubgroup(id=5, name='Semantic Connectivity Layer', description='This layer focuses on the relationships between concepts, ideas, and entities within the text. It involves identifying connections, references, and allusions that contribute to the deeper meaning and understanding of the content.'), CognitiveLayerSubgroup(id=6, name='Rhetorical Strategies Layer', description='This layer scrutinizes the rhetorical techniques and strategies employed by the author to persuade the audience. It looks at the use of ethos, pathos, and logos, and how these contribute to the effectiveness of the argument.'), CognitiveLayerSubgroup(id=7, name='Audience Engagement Layer', description='This layer considers the projected audience and the potential impact or response the text could elicit. It includes analyzing how the text addresses the reader, anticipates their questions or concerns, and engages them in the discourse.'), CognitiveLayerSubgroup(id=8, name='Narrative Flow Layer', description='This layer assesses the flow and progression of ideas throughout the text. It looks at how smoothly the text transitions from one point to another, the pacing of information delivery, and the narrative techniques used to maintain interest.'), CognitiveLayerSubgroup(id=9, name='Cognitive Dissonance Layer', description=\"This layer identifies instances where the text presents ideas or arguments that may challenge the reader's pre-existing beliefs or assumptions. It examines how the text handles possible contradictions and the resolution of conflicting viewpoints.\"), CognitiveLayerSubgroup(id=10, name='Social Influence Layer', description=\"This layer explores the social implications of the text, including the influence on the reader's beliefs, behaviors, and social dynamics. It also considers the potential for the text to reflect or shape societal norms and values.\")]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(cognitive_layers)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 39,
|
||
"id": "1a287a2a-2fb5-4ad3-a69e-80ed2e2ffa5a",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Extracted Layer Names: ['Thematic Content Layer', 'Structural Composition Layer', 'Linguistic Style Layer', 'Contextual Relevance Layer', 'Semantic Connectivity Layer', 'Rhetorical Strategies Layer', 'Audience Engagement Layer', 'Narrative Flow Layer', 'Cognitive Dissonance Layer', 'Social Influence Layer']\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"layer_names = [layer_subgroup.name for layer_subgroup in cognitive_layers.cognitive_layers]\n",
|
||
"\n",
|
||
"print(\"Extracted Layer Names:\", layer_names)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 40,
|
||
"id": "07014c19-e981-4150-afc0-78800062f6e0",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"['Thematic Content Layer',\n",
|
||
" 'Structural Composition Layer',\n",
|
||
" 'Linguistic Style Layer']"
|
||
]
|
||
},
|
||
"execution_count": 40,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"layer_names[:3]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "619a765d-1fa9-49ce-9b6c-66e81a50e409",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "dbce6243-7501-42d1-b944-f80811ae903d",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def system_prompt(layer:str=None)->str: \n",
|
||
" return f\"\"\"You are a top-tier algorithm\n",
|
||
"designed for extracting information in structured formats to build a knowledge graph.\n",
|
||
"- **Nodes** represent entities and concepts. They're akin to Wikipedia nodes.\n",
|
||
"- **Edges** represent relationships between concepts. They're akin to Wikipedia links.\n",
|
||
"- The aim is to achieve simplicity and clarity in the\n",
|
||
"knowledge graph, making it accessible for a vast audience.\n",
|
||
"YOU ARE ONLY EXTRACTING DATA FOR COGNITIVE LAYER {layer}\n",
|
||
"## 2. Labeling Nodes\n",
|
||
"- **Consistency**: Ensure you use basic or elementary types for node labels.\n",
|
||
" - For example, when you identify an entity representing a person,\n",
|
||
" always label it as **\"person\"**.\n",
|
||
" Avoid using more specific terms like \"mathematician\" or \"scientist\".\n",
|
||
" - Include event, entity, time, or action nodes to the category.\n",
|
||
" - Classify the memory type as episodic or semantic.\n",
|
||
"- **Node IDs**: Never utilize integers as node IDs.\n",
|
||
" Node IDs should be names or human-readable identifiers found in the text.\n",
|
||
"## 3. Handling Numerical Data and Dates\n",
|
||
"- Numerical data, like age or other related information,\n",
|
||
"should be incorporated as attributes or properties of the respective nodes.\n",
|
||
"- **No Separate Nodes for Dates/Numbers**:\n",
|
||
"Do not create separate nodes for dates or numerical values.\n",
|
||
" Always attach them as attributes or properties of nodes.\n",
|
||
"- **Property Format**: Properties must be in a key-value format.\n",
|
||
"- **Quotation Marks**: Never use escaped single or double quotes within property values.\n",
|
||
"- **Naming Convention**: Use camelCase for property keys, e.g., `birthDate`.\n",
|
||
"## 4. Coreference Resolution\n",
|
||
"- **Maintain Entity Consistency**:\n",
|
||
"When extracting entities, it's vital to ensure consistency.\n",
|
||
"If an entity, such as \"John Doe\", is mentioned multiple times\n",
|
||
"in the text but is referred to by different names or pronouns (e.g., \"Joe\", \"he\"),\n",
|
||
"always use the most complete identifier for that entity throughout the knowledge graph.\n",
|
||
" In this example, use \"John Doe\" as the entity ID.\n",
|
||
"Remember, the knowledge graph should be coherent and easily understandable,\n",
|
||
" so maintaining consistency in entity references is crucial.\n",
|
||
"## 5. Strict Compliance\n",
|
||
"Adhere to the rules strictly. Non-compliance will result in termination\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "12a0d829-1387-4e32-84a1-1ad7b6edf0dc",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import instructor\n",
|
||
"from openai import OpenAI\n",
|
||
"\n",
|
||
"from dotenv import load_dotenv\n",
|
||
"import os\n",
|
||
"\n",
|
||
"# Load environment variables from .env file\n",
|
||
"load_dotenv()\n",
|
||
"\n",
|
||
"aclient = instructor.patch(OpenAI())\n",
|
||
"\n",
|
||
"from typing import Optional, List\n",
|
||
"from pydantic import BaseModel, Field\n",
|
||
"\n",
|
||
"\n",
|
||
"class Node(BaseModel):\n",
|
||
" \"\"\"Node in a knowledge graph.\"\"\"\n",
|
||
" id: int\n",
|
||
" description: str\n",
|
||
" category: str\n",
|
||
" memory_type: str\n",
|
||
" created_at: Optional[float] = None\n",
|
||
" summarized: Optional[bool] = None\n",
|
||
"\n",
|
||
"\n",
|
||
"class Edge(BaseModel):\n",
|
||
" \"\"\"Edge in a knowledge graph.\"\"\"\n",
|
||
" source: int\n",
|
||
" target: int\n",
|
||
" description: str\n",
|
||
" created_at: Optional[float] = None\n",
|
||
" summarized: Optional[bool] = None\n",
|
||
"\n",
|
||
"\n",
|
||
"class KnowledgeGraph(BaseModel):\n",
|
||
" \"\"\"Knowledge graph.\"\"\"\n",
|
||
" nodes: List[Node] = Field(..., default_factory=list)\n",
|
||
" edges: List[Edge] = Field(..., default_factory=list)\n",
|
||
"\n",
|
||
"\n",
|
||
"def generate_graph(input, layer:str=None) -> KnowledgeGraph:\n",
|
||
" \"\"\"Generate a knowledge graph from a user query.\"\"\"\n",
|
||
" model = \"gpt-4-1106-preview\"\n",
|
||
" user_prompt = f\"Use the given format to extract information from the following input: {input}.\"\n",
|
||
"\n",
|
||
"\n",
|
||
" out = aclient.chat.completions.create(\n",
|
||
" model=model,\n",
|
||
" messages=[\n",
|
||
" {\n",
|
||
" \"role\": \"user\",\n",
|
||
" \"content\": user_prompt,\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"role\": \"system\",\n",
|
||
" \"content\": system_prompt(layer=layer),\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"role\": \"system\",\n",
|
||
" \"content\": \"Must include both nodes and edges\",\n",
|
||
" },\n",
|
||
" ],\n",
|
||
" response_model=KnowledgeGraph,\n",
|
||
" )\n",
|
||
" return out"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 41,
|
||
"id": "12bf38da-019d-4568-af21-21507c60f906",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Layer processed is: Thematic Content Layer\n",
|
||
"Layer graph is: nodes=[Node(id=1, description='Economist who wrote about the preference of people to prove existing beliefs over changing their minds.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Believed that unformed minds can understand difficult subjects but biased minds cannot accept even simple things.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Humans prioritize social belonging over accuracy, causing them to hold beliefs for social advantage rather than truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='A powerful human need which often conflicts with the pursuit of truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='The conflict between social belonging and understanding the truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='Suggests that the brain holds beliefs that bring social alliances over strictly true beliefs.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Notes that the brain is rewarded for adopting certain beliefs regardless of their factual accuracy.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='A term used to describe beliefs that are incorrect but socially beneficial to hold.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='The idea that friendships and social ties have a greater impact on changing minds than factual evidence.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Advocates sharing meals with those of opposing views to promote tolerance and understanding.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='A conceptual framework that suggests engaging with people who hold similar but not identical beliefs to effectively change minds.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='Introduced the idea that minds are more likely to be changed by those we mostly agree with.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The concept that books, due to their private and introspective nature, can facilitate belief change better than public debates.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description='States that the prevalence of an idea is proportional to how much it is repeated, regardless of its truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=15, description='Suggests spending as little time as possible discussing the wrongness of others to starve bad ideas and promote good ones.', category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description=\"Galbraith's quote reflects the logic of false beliefs.\", created_at=None, summarized=None), Edge(source=2, target=3, description=\"Tolstoy's quote supports the ideas about the persistence of false beliefs.\", created_at=None, summarized=None), Edge(source=4, target=3, description='The need for social belonging is central to the logic of false beliefs.', created_at=None, summarized=None), Edge(source=5, target=3, description='Tribalism highlights the conflict in the logic of false beliefs.', created_at=None, summarized=None), Edge(source=6, target=3, description='Pinker provides psychological insight into the logic of false beliefs.', created_at=None, summarized=None), Edge(source=7, target=3, description=\"Simler's views add to the understanding of the logic of false beliefs.\", created_at=None, summarized=None), Edge(source=8, target=3, description='The term explains a key aspect of the logic of false beliefs.', created_at=None, summarized=None), Edge(source=9, target=3, description='This idea illustrates an exception within the logic of false beliefs.', created_at=None, summarized=None), Edge(source=10, target=9, description='de Botton suggests a practical application of friendship over facts.', created_at=None, summarized=None), Edge(source=11, target=9, description=\"The spectrum of beliefs framework supports the 'friendship over facts' idea.\", created_at=None, summarized=None), Edge(source=12, target=11, description=\"Casnocha's idea is at the core of the spectrum of beliefs.\", created_at=None, summarized=None), Edge(source=13, target=11, description='Books serve as an effective way to change beliefs within the spectrum of beliefs.', created_at=None, summarized=None), Edge(source=14, target=3, description=\"Clear's Law outlines a phenomenon relevant to the persistence of false beliefs.\", created_at=None, summarized=None), Edge(source=15, target=14, description=\"Cowen's quote complements Clear's Law of Recurrence with a strategy to diminish bad ideas.\", created_at=None, summarized=None)]\n",
|
||
"Layer processed is: Structural Composition Layer\n",
|
||
"Layer graph is: nodes=[Node(id=1, description='The economist J.K. Galbraith', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='The writer Leo Tolstoy', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='The phenomenon of belief persistence despite contradicting facts', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='The psychology behind preferring social belonging over factual correctness', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description=\"The book 'Atomic Habits' discussing humans as herd animals and the need to fit in\", category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='The Harvard psychologist Steven Pinker', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='The concept that rewards from peers encourage belief adoption', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Kevin Simler, a writer who discussed the influence of rewards on belief adoption', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='The idea that friendship and social connections have a stronger impact on belief change than facts', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The British philosopher Alain de Botton', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='The concept of a spectrum of beliefs and the likelihood of changing minds', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='Ben Casnocha, a person who introduced the idea of the spectrum of beliefs', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The notion that books are more effective than conversations for belief change', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description='The persistence of false ideas due to their continued discussion', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=15, description=\"Clear's Law of Recurrence, posited by James Clear\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=16, description=\"Tyler Cowen, an economist who comments on the nature of discussion about others' wrongs\", category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description=\"discussed the concept of proof over changing one's mind\", created_at=None, summarized=None), Edge(source=2, target=3, description='commented on the explanation of subjects relative to preconceived notions', created_at=None, summarized=None), Edge(source=5, target=4, description='introduces the concept of human sociability and tribe belonging', created_at=None, summarized=None), Edge(source=6, target=4, description='explains the social aspect of belief adoption', created_at=None, summarized=None), Edge(source=8, target=7, description='discusses the reward system in brain related to beliefs', created_at=None, summarized=None), Edge(source=10, target=9, description='suggests sharing meals to bridge differences', created_at=None, summarized=None), Edge(source=12, target=11, description='introduced idea to author James Clear about spectrum of beliefs', created_at=None, summarized=None), Edge(source=16, target=14, description=\"advises less focus on discussing others' wrongs\", created_at=None, summarized=None)]\n",
|
||
"Layer processed is: Linguistic Style Layer\n",
|
||
"Layer graph is: nodes=[Node(id=1, description='The economist who wrote about the challenges people face when having to change their minds.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='The writer who suggested that unformed ideas can be explained to anyone regardless of intelligence', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Have a desire to belong and maintain social connections, sometimes at the cost of accepting false beliefs.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Harvard psychologist who spoke about the social function of beliefs in relation to acceptance in society.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description=\"Wrote about the brain's willingness to adopt beliefs for rewards, whether pragmatic or social.\", category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description=\"Often less influential than social connections in changing a person's mind or beliefs.\", category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='A more effective way to change minds than presenting facts, as it involves social integration.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='British philosopher who suggests that sharing meals can promote tolerance and reduce prejudice.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='A concept where beliefs are visualized on a spectrum, showing that nearby beliefs are more easily influenced.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Described as a better medium for transforming beliefs as they allow ideas to be considered without social pressures.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='The idea that the more an idea is repeated, the more it is believed, regardless of its veracity.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description=\"Advocates spending minimal time on highlighting others' mistakes and instead focusing on positive ideas.\", category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='A book that discusses human inclinations to belong, bond, and be accepted by others.', category='entity', memory_type='episodic', created_at=None, summarized=None), Node(id=14, description=\"Author of 'Atomic Habits', discusses the logic of false beliefs and the importance of social connections over facts.\", category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description='discussed challenges faced by humans when changing their minds', created_at=None, summarized=None), Edge(source=2, target=3, description='suggested that preconceived notions hinder understanding simple concepts', created_at=None, summarized=None), Edge(source=3, target=4, description='Steven Pinker provides insight on human beliefs and their social function', created_at=None, summarized=None), Edge(source=3, target=5, description=\"Kevin Simler wrote about human brain's response towards adopting beliefs for various rewards\", created_at=None, summarized=None), Edge(source=3, target=6, description='tend to prioritize social connections over facts when changing beliefs', created_at=None, summarized=None), Edge(source=6, target=7, description=\"less influential than friendship in changing people's beliefs\", created_at=None, summarized=None), Edge(source=7, target=8, description='Alain de Botton suggests, through friendship, shared meals can help bridge differences', created_at=None, summarized=None), Edge(source=3, target=9, description='are more inclined to change their beliefs when influenced by those they are socially or ideologically close to', created_at=None, summarized=None), Edge(source=3, target=10, description='find books to be a non-threatening environment for considering new ideas', created_at=None, summarized=None), Edge(source=3, target=11, description=\"tend to believe ideas that are repeatedly mentioned, as described in Clear's Law of Recurrence\", created_at=None, summarized=None), Edge(source=12, target=11, description=\"emphasizes not to spend time on bad ideas, aligning with Clear's Law of Recurrence\", created_at=None, summarized=None), Edge(source=14, target=13, description='wrote the book discussing human social inclinations', created_at=None, summarized=None)]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"layer_graphs = []\n",
|
||
"\n",
|
||
"for layer in layer_names[:3]:\n",
|
||
" print(\"Layer processed is:\", str(layer))\n",
|
||
" layer_graph = generate_graph(input=input, layer= layer)\n",
|
||
" print(\"Layer graph is:\", str(layer_graph))\n",
|
||
" layer_graphs.append(layer_graph)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"id": "4a19cc82-b892-47f3-99db-b70edccefda5",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"KnowledgeGraph(nodes=[Node(id=1, description='J.K. Galbraith', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Leo Tolstoy', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Steven Pinker', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Kevin Simler', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='Alain de Botton', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='Abraham Lincoln', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Ben Casnocha', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Tyler Cowen', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Atomic Habits - Book', category='creativeWork', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The logic and benefits of holding false beliefs', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description=\"Facts don't change our minds, friendship does\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='The spectrum of beliefs and changeability', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The persistence of false ideas through recurrence', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description=\"Clear's Law of Recurrence\", category='concept', memory_type='semantic', created_at=None, summarized=None)], edges=[Edge(source=1, target=10, description=\"J.K. Galbraith wrote about the human tendency to prove existing beliefs rather than change one's mind.\", created_at=None, summarized=None), Edge(source=2, target=10, description='Leo Tolstoy noted the difficulty of explaining things to someone who is certain in their pre-existing beliefs.', created_at=None, summarized=None), Edge(source=9, target=10, description='Atomic Habits discusses the importance of social belonging over accuracy of beliefs.', created_at=None, summarized=None), Edge(source=3, target=10, description='Steven Pinker suggests that beliefs may function to secure social allies over truth.', created_at=None, summarized=None), Edge(source=4, target=10, description=\"Kevin Simler described beliefs as anticipatory rewards, with little care for the reward's origin.\", created_at=None, summarized=None), Edge(source=5, target=11, description='Alain de Botton proposed sharing meals to bridge differences and foster understanding.', created_at=None, summarized=None), Edge(source=6, target=11, description=\"Abraham Lincoln's quote highlights the importance of understanding others to overcome dislike.\", created_at=None, summarized=None), Edge(source=7, target=12, description='Ben Casnocha introduced the idea that those we largely agree with are most likely to change our minds.', created_at=None, summarized=None), Edge(source=14, target=13, description=\"Clear's Law of Recurrence states that repeated ideas, even if false, gain belief proportionally to their repetition.\", created_at=None, summarized=None), Edge(source=8, target=13, description='Tyler Cowen advised focusing on good ideas rather than disputing wrong ones.', created_at=None, summarized=None)])"
|
||
]
|
||
},
|
||
"execution_count": 18,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"layer_decomposition"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "15dc7863-0f4c-47ae-89ef-2656e8478249",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"id": "58644c64-7ef0-415f-8e41-e2edcf5fd15b",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import networkx as nx\n",
|
||
"import uuid\n",
|
||
"from datetime import datetime\n",
|
||
"\n",
|
||
"def create_user_content_graph(user_id, custom_user_properties=None, additional_categories=None, default_fields=None):\n",
|
||
" # Define default fields for all nodes if not provided\n",
|
||
" if default_fields is None:\n",
|
||
" default_fields = {\n",
|
||
" 'created_at': datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\"),\n",
|
||
" 'updated_at': datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\")\n",
|
||
" }\n",
|
||
"\n",
|
||
" # Merge custom user properties with default properties; custom properties take precedence\n",
|
||
" user_properties = {**default_fields, **(custom_user_properties or {})}\n",
|
||
"\n",
|
||
" # Default content categories\n",
|
||
" content_categories = {\n",
|
||
" \"Temporal\": [\"Historical events\", \"Schedules and timelines\"],\n",
|
||
" \"Positional\": [\"Geographical locations\", \"Spatial data\"],\n",
|
||
" \"Propositions\": [\"Hypotheses and theories\", \"Claims and arguments\"],\n",
|
||
" \"Personalization\": [\"User preferences\", \"User information\"]\n",
|
||
" }\n",
|
||
"\n",
|
||
" # Update content categories with any additional categories provided\n",
|
||
" if additional_categories:\n",
|
||
" content_categories.update(additional_categories)\n",
|
||
"\n",
|
||
" # Create a new MultiDiGraph\n",
|
||
" G = nx.MultiDiGraph()\n",
|
||
"\n",
|
||
" # Add the user node with properties\n",
|
||
" G.add_node(user_id, **user_properties)\n",
|
||
"\n",
|
||
" # Add content category nodes and their edges\n",
|
||
" for category, subclasses in content_categories.items():\n",
|
||
" category_properties = {**default_fields, 'type': 'category'}\n",
|
||
" G.add_node(category, **category_properties)\n",
|
||
" G.add_edge(user_id, category, relationship='created')\n",
|
||
"\n",
|
||
" # Add subclass nodes and their edges\n",
|
||
" for subclass in subclasses:\n",
|
||
" unique_id = str(uuid.uuid4())\n",
|
||
" subclass_node_id = f\"{subclass} - {unique_id}\"\n",
|
||
" subclass_properties = {**default_fields, 'type': 'subclass', 'content': subclass}\n",
|
||
" G.add_node(subclass_node_id, **subclass_properties)\n",
|
||
" G.add_edge(category, subclass_node_id, relationship='includes')\n",
|
||
"\n",
|
||
" return G\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 20,
|
||
"id": "dd3f0e55-9f9d-4804-9ad6-31afd2088ab5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"category_name = required_layers.dict()['name']\n",
|
||
"\n",
|
||
"# Extract the names of the cognitive subgroups\n",
|
||
"subgroup_names = [subgroup['name'] for subgroup in required_layers.dict()['cognitive_subgroups']]\n",
|
||
"\n",
|
||
"# Construct the additional_categories structure\n",
|
||
"additional_categories = {\n",
|
||
" category_name: subgroup_names\n",
|
||
"}\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 44,
|
||
"id": "2cc7c3bb-7cc0-453b-beab-2983a703ccda",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"{'Natural Language Text': ['Articles, essays, and reports']}"
|
||
]
|
||
},
|
||
"execution_count": 44,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"additional_categories"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 21,
|
||
"id": "199ef3ab-5e73-40d2-b531-6a402edf3f17",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Nodes in the graph:\n",
|
||
"[('user123', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'username': 'exampleUser', 'email': 'user@example.com'}), ('Temporal', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Historical events - fe2b09db-93cb-4763-b186-5e08aa83b2f0', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Historical events'}), ('Schedules and timelines - 9584f7c2-bb9e-4b8d-9bf5-78545e705648', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Schedules and timelines'}), ('Positional', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Geographical locations - a695710f-9a71-4949-81a4-773e7c693802', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Geographical locations'}), ('Spatial data - 8ee3d6fb-1919-4cdb-978a-fe27d1456cd5', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Spatial data'}), ('Propositions', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Hypotheses and theories - 9e8d48b9-0a75-47f5-a113-b5c28cf6484d', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Hypotheses and theories'}), ('Claims and arguments - 0d6d809c-5858-4998-a959-3cd9494f5e24', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Claims and arguments'}), ('Personalization', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('User preferences - f84f9b30-42dd-47aa-bac0-170442371a80', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'User preferences'}), ('User information - 067b9536-8ce2-4cd2-ae9b-ca0771e79580', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'User information'}), ('Natural Language Text', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Articles, essays, and reports - 97841c38-3dc7-4964-8b0c-ba8875a3b4ea', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Articles, essays, and reports'})]\n",
|
||
"\n",
|
||
"Edges in the graph:\n",
|
||
"[('user123', 'Temporal', {'relationship': 'created'}), ('user123', 'Positional', {'relationship': 'created'}), ('user123', 'Propositions', {'relationship': 'created'}), ('user123', 'Personalization', {'relationship': 'created'}), ('user123', 'Natural Language Text', {'relationship': 'created'}), ('Temporal', 'Historical events - fe2b09db-93cb-4763-b186-5e08aa83b2f0', {'relationship': 'includes'}), ('Temporal', 'Schedules and timelines - 9584f7c2-bb9e-4b8d-9bf5-78545e705648', {'relationship': 'includes'}), ('Positional', 'Geographical locations - a695710f-9a71-4949-81a4-773e7c693802', {'relationship': 'includes'}), ('Positional', 'Spatial data - 8ee3d6fb-1919-4cdb-978a-fe27d1456cd5', {'relationship': 'includes'}), ('Propositions', 'Hypotheses and theories - 9e8d48b9-0a75-47f5-a113-b5c28cf6484d', {'relationship': 'includes'}), ('Propositions', 'Claims and arguments - 0d6d809c-5858-4998-a959-3cd9494f5e24', {'relationship': 'includes'}), ('Personalization', 'User preferences - f84f9b30-42dd-47aa-bac0-170442371a80', {'relationship': 'includes'}), ('Personalization', 'User information - 067b9536-8ce2-4cd2-ae9b-ca0771e79580', {'relationship': 'includes'}), ('Natural Language Text', 'Articles, essays, and reports - 97841c38-3dc7-4964-8b0c-ba8875a3b4ea', {'relationship': 'includes'})]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Example usage\n",
|
||
"user_id = 'user123'\n",
|
||
"custom_user_properties = {\n",
|
||
" 'username': 'exampleUser',\n",
|
||
" 'email': 'user@example.com'\n",
|
||
"}\n",
|
||
"\n",
|
||
"# additional_categories = {\n",
|
||
"# \"Natural Language Text\": [\"Articles, essays, and reports\", \"Books and manuscripts\"]\n",
|
||
"# }\n",
|
||
"\n",
|
||
"G = create_user_content_graph(user_id, custom_user_properties, additional_categories)\n",
|
||
"\n",
|
||
"# Accessing the graph\n",
|
||
"print(\"Nodes in the graph:\")\n",
|
||
"print(G.nodes(data=True))\n",
|
||
"print(\"\\nEdges in the graph:\")\n",
|
||
"print(G.edges(data=True))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "4dab2ff0-0d12-4a00-a4e4-fb901e701bd3",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 22,
|
||
"id": "512f15be-0114-4c8c-9754-e82f2fa16344",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"\n",
|
||
" <iframe id=\"68f2d96b-7351-4426-9d3a-2914680f8a9a\" src=\"https://hub.graphistry.com/graph/graph.html?dataset=c054a3e2591b4ad7b1ad2b0b35e3e9d3&type=arrow&viztoken=7779f415-c787-4459-ac14-73cb9f247f04&usertag=1daaf574-pygraphistry-0.33.0&splashAfter=1708878372&info=true\"\n",
|
||
" allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
|
||
" oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
|
||
" style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\"\n",
|
||
" \n",
|
||
" >\n",
|
||
" </iframe>\n",
|
||
" \n",
|
||
" <script>\n",
|
||
" try {\n",
|
||
" $(\"#68f2d96b-7351-4426-9d3a-2914680f8a9a\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
|
||
" } catch (e) { console.error('exn catching scroll', e); }\n",
|
||
" </script>\n",
|
||
" "
|
||
],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"execution_count": 22,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"import graphistry\n",
|
||
"import pandas as pd\n",
|
||
"\n",
|
||
"# Assuming Graphistry is already configured with API key\n",
|
||
"# graphistry.register(api=3, username='your_username', password='your_password')\n",
|
||
"\n",
|
||
"# Convert NetworkX graph to a Pandas DataFrame\n",
|
||
"edges = nx.to_pandas_edgelist(G)\n",
|
||
"graphistry.register(api=3, username='Vasilije1990', password='Q@HLdgv5SMUsGxy') \n",
|
||
"\n",
|
||
"# Visualize the graph\n",
|
||
"graphistry.edges(edges, 'source', 'target').plot()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 23,
|
||
"id": "40866ff1-7c2d-4682-851c-de2442984cd5",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"MultiDiGraph with 15 nodes and 14 edges\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(G)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 24,
|
||
"id": "a3127724-3bef-416e-8631-ba6cea645196",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"KnowledgeGraph(nodes=[Node(id=1, description='J.K. Galbraith', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Leo Tolstoy', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Steven Pinker', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Kevin Simler', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='Alain de Botton', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='Abraham Lincoln', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Ben Casnocha', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Tyler Cowen', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Atomic Habits - Book', category='creativeWork', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The logic and benefits of holding false beliefs', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description=\"Facts don't change our minds, friendship does\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='The spectrum of beliefs and changeability', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The persistence of false ideas through recurrence', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description=\"Clear's Law of Recurrence\", category='concept', memory_type='semantic', created_at=None, summarized=None)], edges=[Edge(source=1, target=10, description=\"J.K. Galbraith wrote about the human tendency to prove existing beliefs rather than change one's mind.\", created_at=None, summarized=None), Edge(source=2, target=10, description='Leo Tolstoy noted the difficulty of explaining things to someone who is certain in their pre-existing beliefs.', created_at=None, summarized=None), Edge(source=9, target=10, description='Atomic Habits discusses the importance of social belonging over accuracy of beliefs.', created_at=None, summarized=None), Edge(source=3, target=10, description='Steven Pinker suggests that beliefs may function to secure social allies over truth.', created_at=None, summarized=None), Edge(source=4, target=10, description=\"Kevin Simler described beliefs as anticipatory rewards, with little care for the reward's origin.\", created_at=None, summarized=None), Edge(source=5, target=11, description='Alain de Botton proposed sharing meals to bridge differences and foster understanding.', created_at=None, summarized=None), Edge(source=6, target=11, description=\"Abraham Lincoln's quote highlights the importance of understanding others to overcome dislike.\", created_at=None, summarized=None), Edge(source=7, target=12, description='Ben Casnocha introduced the idea that those we largely agree with are most likely to change our minds.', created_at=None, summarized=None), Edge(source=14, target=13, description=\"Clear's Law of Recurrence states that repeated ideas, even if false, gain belief proportionally to their repetition.\", created_at=None, summarized=None), Edge(source=8, target=13, description='Tyler Cowen advised focusing on good ideas rather than disputing wrong ones.', created_at=None, summarized=None)])"
|
||
]
|
||
},
|
||
"execution_count": 24,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"layer_decomposition "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 35,
|
||
"id": "fe634bcb-0c00-4a2a-8bcb-687a2fcf847c",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Updated Nodes: MultiDiGraph with 43 nodes and 52 edges\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def append_data_to_graph(G, category_name, subclass_content, new_data):\n",
|
||
" # Find the node ID for the subclass within the category\n",
|
||
" subclass_node_id = None\n",
|
||
" for node, data in G.nodes(data=True):\n",
|
||
" if subclass_content in node:\n",
|
||
" subclass_node_id = node\n",
|
||
" break\n",
|
||
"\n",
|
||
" if not subclass_node_id:\n",
|
||
" print(f\"Subclass '{subclass_content}' under category '{category_name}' not found in the graph.\")\n",
|
||
" return G\n",
|
||
"\n",
|
||
" # Mapping from old node IDs to new node IDs\n",
|
||
" node_id_mapping = {}\n",
|
||
"\n",
|
||
" # Add nodes from the Pydantic object\n",
|
||
" for node in new_data.nodes:\n",
|
||
" new_node_id = f\"{subclass_node_id} - {str(uuid.uuid4())}\"\n",
|
||
" G.add_node(new_node_id, \n",
|
||
" created_at=datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\"), \n",
|
||
" updated_at=datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\"), \n",
|
||
" description=node.description, \n",
|
||
" category=node.category, \n",
|
||
" memory_type=node.memory_type, \n",
|
||
" type='detail')\n",
|
||
" G.add_edge(subclass_node_id, new_node_id, relationship='detail')\n",
|
||
"\n",
|
||
" # Store the mapping from old node ID to new node ID\n",
|
||
" node_id_mapping[node.id] = new_node_id\n",
|
||
"\n",
|
||
" # Add edges from the Pydantic object using the new node IDs\n",
|
||
" for edge in new_data.edges:\n",
|
||
" # Use the mapping to get the new node IDs\n",
|
||
" source_node_id = node_id_mapping.get(edge.source)\n",
|
||
" target_node_id = node_id_mapping.get(edge.target)\n",
|
||
"\n",
|
||
" if source_node_id and target_node_id:\n",
|
||
" G.add_edge(source_node_id, target_node_id, description=edge.description, relationship='relation')\n",
|
||
" else:\n",
|
||
" print(f\"Could not find mapping for edge from {edge.source} to {edge.target}\")\n",
|
||
"\n",
|
||
" return G\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"# Assuming `pydata` is your Pydantic model instance containing the nodes and edges information\n",
|
||
"# and `G` is your existing graph\n",
|
||
"\n",
|
||
"# Here's how you would call this function:\n",
|
||
"category_name = list(additional_categories.keys())[0]\n",
|
||
"subclass_content = list(additional_categories.values())[0][0]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 42,
|
||
"id": "66630223-3ba0-4384-95d2-df2995e15271",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Updated Nodes: MultiDiGraph with 58 nodes and 81 edges\n",
|
||
"Updated Nodes: MultiDiGraph with 74 nodes and 105 edges\n",
|
||
"Updated Nodes: MultiDiGraph with 88 nodes and 131 edges\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"for layer_decomposition in layer_graphs:\n",
|
||
" F = append_data_to_graph(G, category_name, subclass_content, layer_decomposition)\n",
|
||
" \n",
|
||
" # Print updated graph for verification\n",
|
||
" print(\"Updated Nodes:\", F)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 43,
|
||
"id": "17199837-35b8-4530-bf03-efbc3486b71d",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"\n",
|
||
" <iframe id=\"92958360-335e-4f3a-912f-a2f8c65905e7\" src=\"https://hub.graphistry.com/graph/graph.html?dataset=30b3be0fec7645108e25304e5b575d26&type=arrow&viztoken=91f96a9c-4cee-4ca5-8411-d5b429971f18&usertag=1daaf574-pygraphistry-0.33.0&splashAfter=1708931229&info=true\"\n",
|
||
" allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"\n",
|
||
" oallowfullscreen=\"true\" msallowfullscreen=\"true\"\n",
|
||
" style=\"width:100%; height:500px; border: 1px solid #DDD; overflow: hidden\"\n",
|
||
" \n",
|
||
" >\n",
|
||
" </iframe>\n",
|
||
" \n",
|
||
" <script>\n",
|
||
" try {\n",
|
||
" $(\"#92958360-335e-4f3a-912f-a2f8c65905e7\").bind('mousewheel', function(e) { e.preventDefault(); });\n",
|
||
" } catch (e) { console.error('exn catching scroll', e); }\n",
|
||
" </script>\n",
|
||
" "
|
||
],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"execution_count": 43,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"import graphistry\n",
|
||
"import pandas as pd\n",
|
||
"\n",
|
||
"# Assuming Graphistry is already configured with API key\n",
|
||
"# graphistry.register(api=3, username='your_username', password='your_password')\n",
|
||
"\n",
|
||
"# Convert NetworkX graph to a Pandas DataFrame\n",
|
||
"edges = nx.to_pandas_edgelist(F)\n",
|
||
"graphistry.register(api=3, username='Vasilije1990', password='') \n",
|
||
"\n",
|
||
"# Visualize the graph\n",
|
||
"graphistry.edges(edges, 'source', 'target').plot()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "187f4220-45b1-43cc-9ab3-1e51264544c5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 40,
|
||
"id": "f17dc174-f325-44b9-8fc7-761146aca2d0",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Articles, essays, and reports\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(list(additional_categories.values())[0][0])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 42,
|
||
"id": "e4eee8a7-5d3b-4848-9cdb-2b397e158519",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'Natural Language Text'"
|
||
]
|
||
},
|
||
"execution_count": 42,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"list(additional_categories.keys())[0]"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 37,
|
||
"id": "3adc8483-3207-44f1-abf5-275e925e04d4",
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Graph Relationships with Node Attributes:\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, username: exampleUser, email: user@example.com] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category]: Relationship [created]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, username: exampleUser, email: user@example.com] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category]: Relationship [created]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, username: exampleUser, email: user@example.com] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category]: Relationship [created]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, username: exampleUser, email: user@example.com] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category]: Relationship [created]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, username: exampleUser, email: user@example.com] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category]: Relationship [created]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Historical events]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Schedules and timelines]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Geographical locations]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Spatial data]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Hypotheses and theories]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Claims and arguments]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: User preferences]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: User information]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: category] -> Target [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports]: Relationship [includes]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: J.K. Galbraith, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Leo Tolstoy, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Steven Pinker, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Kevin Simler, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Alain de Botton, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Abraham Lincoln, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Ben Casnocha, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Tyler Cowen, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Atomic Habits - Book, category: creativeWork, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Facts don't change our minds, friendship does, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: The spectrum of beliefs and changeability, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: The persistence of false ideas through recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:25:57, updated_at: 2024-02-25 17:25:57, description: Clear's Law of Recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: J.K. Galbraith, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Leo Tolstoy, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Steven Pinker, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Kevin Simler, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Alain de Botton, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Abraham Lincoln, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Ben Casnocha, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Tyler Cowen, category: person, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Atomic Habits - Book, category: creativeWork, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Facts don't change our minds, friendship does, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The spectrum of beliefs and changeability, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The persistence of false ideas through recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:25:52, updated_at: 2024-02-25 17:25:52, type: subclass, content: Articles, essays, and reports] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Clear's Law of Recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [detail]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: J.K. Galbraith, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Leo Tolstoy, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Steven Pinker, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Kevin Simler, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Alain de Botton, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Facts don't change our minds, friendship does, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Abraham Lincoln, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Facts don't change our minds, friendship does, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Ben Casnocha, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The spectrum of beliefs and changeability, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Tyler Cowen, category: person, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The persistence of false ideas through recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Atomic Habits - Book, category: creativeWork, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The logic and benefits of holding false beliefs, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n",
|
||
"Source [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: Clear's Law of Recurrence, category: concept, memory_type: semantic, type: detail] -> Target [created_at: 2024-02-25 17:34:05, updated_at: 2024-02-25 17:34:05, description: The persistence of false ideas through recurrence, category: concept, memory_type: semantic, type: detail]: Relationship [relation]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def list_graph_relationships_with_node_attributes(graph):\n",
|
||
" print(\"Graph Relationships with Node Attributes:\")\n",
|
||
" for source, target, data in graph.edges(data=True):\n",
|
||
" # Get source and target node attributes\n",
|
||
" source_attrs = graph.nodes[source]\n",
|
||
" target_attrs = graph.nodes[target]\n",
|
||
" relationship = data.get('relationship', 'No relationship specified')\n",
|
||
"\n",
|
||
" # Format and print source and target node attributes along with the relationship\n",
|
||
" source_attrs_formatted = ', '.join([f\"{k}: {v}\" for k, v in source_attrs.items()])\n",
|
||
" target_attrs_formatted = ', '.join([f\"{k}: {v}\" for k, v in target_attrs.items()])\n",
|
||
" \n",
|
||
" print(f\"Source [{source_attrs_formatted}] -> Target [{target_attrs_formatted}]: Relationship [{relationship}]\")\n",
|
||
"\n",
|
||
"# Assuming 'F' is your graph instance\n",
|
||
"list_graph_relationships_with_node_attributes(F)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 45,
|
||
"id": "734a1d1d-a583-4d3f-a1e2-a33f339bfaf0",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 46,
|
||
"id": "1ace3c0f-93f3-47aa-81d6-0e3297971b5a",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"/Users/vasa/Projects/cognee/cognee\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(os.getcwd())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "74b65ea2-b325-4bed-8286-0f2030462794",
|
||
"metadata": {},
|
||
"source": [
|
||
"## HOW TO QUERY THE GRAPH"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "7adbc416-bca9-47e2-9bfc-7ab5de5aeabf",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.6"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|