diff --git a/Demo_graph.ipynb b/Demo_graph.ipynb index fea77e3e5..9d6a4e23b 100644 --- a/Demo_graph.ipynb +++ b/Demo_graph.ipynb @@ -10,7 +10,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 76, "id": "8a8942b5-91d6-4746-b35d-00f58bc16d7b", "metadata": {}, "outputs": [], @@ -42,85 +42,40 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 143, "id": "14484e25-fae8-4306-b03f-dae91fe5d0aa", "metadata": {}, "outputs": [], "source": [ - "input = \"\"\" The economist J.K. Galbraith once wrote, “Faced with a choice between changing one’s mind and proving there is no need to do so, almost everyone gets busy with the proof.”\n", + "input_article_one = \"\"\" n the nicest possible way, Britons have always been a bit silly about animals. “Keeping pets, for the English, is not so much a leisure activity as it is an entire way of life,” wrote the anthropologist Kate Fox in Watching the English, nearly 20 years ago. Our dogs, in particular, have been an acceptable outlet for emotions and impulses we otherwise keep strictly controlled – our latent desire to be demonstratively affectionate, to be silly and chat to strangers. If this seems like an exaggeration, consider the different reactions you’d get if you struck up a conversation with someone in a park with a dog, versus someone on the train.\n", "\n", - "Leo Tolstoy was even bolder: “The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him.”\n", + "Indeed, British society has been set up to accommodate these four-legged ambassadors. In the UK – unlike Australia, say, or New Zealand – dogs are not just permitted on public transport but often openly encouraged. Many pubs and shops display waggish signs, reading, “Dogs welcome, people tolerated”, and have treat jars on their counters. The other day, as I was waiting outside a cafe with a friend’s dog, the barista urged me to bring her inside.\n", "\n", - "What’s going on here? Why don’t facts change our minds? And why would someone continue to believe a false or inaccurate idea anyway? How do such behaviors serve us?\n", - "The Logic of False Beliefs\n", - "Humans need a reasonably accurate view of the world in order to survive. If your model of reality is wildly different from the actual world, then you struggle to take effective actions each day.\n", + "For years, Britons’ non-partisan passion for animals has been consistent amid dwindling common ground. But lately, rather than bringing out the best in us, our relationship with dogs is increasingly revealing us at our worst – and our supposed “best friends” are paying the price.\n", "\n", - "However, truth and accuracy are not the only things that matter to the human mind. Humans also seem to have a deep desire to belong.\n", + "As with so many latent traits in the national psyche, it all came unleashed with the pandemic, when many people thought they might as well make the most of all that time at home and in local parks with a dog. Between 2019 and 2022, the number of pet dogs in the UK rose from about nine million to 13 million. But there’s long been a seasonal surge around this time of year, substantial enough for the Dogs Trust charity to coin its famous slogan back in 1978: “A dog is for life, not just for Christmas.”\n", "\n", - "In Atomic Habits, I wrote, “Humans are herd animals. We want to fit in, to bond with others, and to earn the respect and approval of our peers. Such inclinations are essential to our survival. For most of our evolutionary history, our ancestors lived in tribes. Becoming separated from the tribe—or worse, being cast out—was a death sentence.”\n", + "Green spaces, meanwhile, have been steadily declining, and now many of us have returned to the office, just as those “pandemic dogs” are entering their troublesome teens. It’s a combustible combination and we are already seeing the results: the number of dog attacks recorded by police in England and Wales rose by more than a third between 2018 and 2022.\n", "\n", - "Understanding the truth of a situation is important, but so is remaining part of a tribe. While these two desires often work well together, they occasionally come into conflict.\n", + "At the same time, sites such as Pets4Homes.co.uk are replete with listings for dogs that, their owners accept “with deep regret”, are no longer suited to their lifestyles now that lockdown is over. It may have felt as if it would go on for ever, but was there ever any suggestion it was going to last the average dog’s lifespan of a decade?\n", "\n", - "In many circumstances, social connection is actually more helpful to your daily life than understanding the truth of a particular fact or idea. The Harvard psychologist Steven Pinker put it this way, “People are embraced or condemned according to their beliefs, so one function of the mind may be to hold beliefs that bring the belief-holder the greatest number of allies, protectors, or disciples, rather than beliefs that are most likely to be true.”\n", + "Living beings are being downgraded to mere commodities. You can see it reflected the “designer” breeds currently in fashion, the French bulldogs and pugs that look cute but spend their entire lives in discomfort. American XL bully dogs, now so controversial, are often sought after as a signifier of masculinity: roping an entire other life in service of our egos. Historically, many of Britain’s most popular breeds evolved to hunt vermin, retrieve game, herd, or otherwise do a specific job alongside humans; these days we are breeding and buying them for their aesthetic appeal.\n", "\n", - "We don’t always believe things because they are correct. Sometimes we believe things because they make us look good to the people we care about.\n", + "Underpinning this is a shift to what was long disdained as the “American” approach: treating pets as substitutes for children. In the past in Britain, dogs were treasured on their own terms, for the qualities that made them dogs, and as such, sometimes better than people: their friendliness and trustingness and how they opened up the world for us. They were indulged, certainly – by allowing them on to the sofa or in our beds, for instance, when we’d sworn we never would – but in ways that did not negate or deny their essential otherness.\n", "\n", - "I thought Kevin Simler put it well when he wrote, “If a brain anticipates that it will be rewarded for adopting a particular belief, it’s perfectly happy to do so, and doesn’t much care where the reward comes from — whether it’s pragmatic (better outcomes resulting from better decisions), social (better treatment from one’s peers), or some mix of the two.”\n", + "Now we have more dogs of such ludicrous proportions, they struggle to function as dogs at all – and we treat them accordingly, indulging them as we would ourselves: by buying unnecessary things. The total spend on pets in the UK has more than doubled in the past decade, reaching nearly £10bn last year. That huge rise has not just come from essentials: figures from the marketing agency Mintel suggest that one in five UK owners like their pet to “keep up with the latest trends” in grooming or, heaven forbid, outfits.\n", "\n", - "False beliefs can be useful in a social sense even if they are not useful in a factual sense. For lack of a better phrase, we might call this approach “factually false, but socially accurate.” When we have to choose between the two, people often select friends and family over facts.\n", + "These days pet “boutiques” – like the one that recently opened on my street in Norwich, selling “cold-pressed” dog treats, “paw and nose balms” and spa services – are a widespread sign of gentrification. But it’s not just wealthier areas: this summer in Great Yarmouth, one of the most deprived towns in the country, I noticed seaside stalls selling not one but two brands of ice-cream for dogs.\n", "\n", - "This insight not only explains why we might hold our tongue at a dinner party or look the other way when our parents say something offensive, but also reveals a better way to change the minds of others.\n", + "It suggests dog-lovers have become untethered from their companions’ desires, let alone their needs. Let’s be honest: most dogs would be thrilled to bits to be eating a paper bag, or even their own faeces. And although they are certainly delighted by ice-cream, they don’t need it. But the ways we ourselves find solace – in consumption, by indulging our simian “treat brain” with things that we don’t need and/or aren’t good for us – we have simply extended to our pets.\n", "\n", - "Facts Don’t Change Our Minds. Friendship Does.\n", - "Convincing someone to change their mind is really the process of convincing them to change their tribe. If they abandon their beliefs, they run the risk of losing social ties. You can’t expect someone to change their mind if you take away their community too. You have to give them somewhere to go. Nobody wants their worldview torn apart if loneliness is the outcome.\n", + "It’s hard not to see the rise in dog-friendly restaurants, cinema screenings and even churches as similar to the ludicrous expenditure: a way to placate the two-legged being on the end of the lead (regardless of the experience of others in the vicinity).\n", "\n", - "The way to change people’s minds is to become friends with them, to integrate them into your tribe, to bring them into your circle. Now, they can change their beliefs without the risk of being abandoned socially.\n", + "Meanwhile, many dogs suffer daily deprivation, their worlds made small and monotonous by our busy modern schedules. These are social animals: it’s not natural for them to live without other dogs, let alone in an empty house for eight hours a day, Monday to Friday. If we are besieged by badly behaved dogs, the cause isn’t hard to pinpoint. Many behavioural problems can be alleviated and even addressed by sufficient exercise, supervision and consistent routines, but instead of organising our lives so that our pets may thrive, we show our love with a Halloween-themed cookie, or a new outfit for Instagram likes.\n", "\n", - "The British philosopher Alain de Botton suggests that we simply share meals with those who disagree with us:\n", + "It’s easy to forget that we are sharing our homes with a descendant of the wolf when it is dressed in sheep’s clothing; but the more we learn about animals, the clearer it becomes that our treatment of them, simultaneously adoring and alienated, means they are leading strange, unsatisfying simulacra of the lives they ought to lead.\n", "\n", - "“Sitting down at a table with a group of strangers has the incomparable and odd benefit of making it a little more difficult to hate them with impunity. Prejudice and ethnic strife feed off abstraction. However, the proximity required by a meal – something about handing dishes around, unfurling napkins at the same moment, even asking a stranger to pass the salt – disrupts our ability to cling to the belief that the outsiders who wear unusual clothes and speak in distinctive accents deserve to be sent home or assaulted. For all the large-scale political solutions which have been proposed to salve ethnic conflict, there are few more effective ways to promote tolerance between suspicious neighbours than to force them to eat supper together.”\n", - "\n", - "Perhaps it is not difference, but distance that breeds tribalism and hostility. As proximity increases, so does understanding. I am reminded of Abraham Lincoln’s quote, “I don’t like that man. I must get to know him better.”\n", - "\n", - "Facts don’t change our minds. Friendship does.\n", - "\n", - "The Spectrum of Beliefs\n", - "Years ago, Ben Casnocha mentioned an idea to me that I haven’t been able to shake: The people who are most likely to change our minds are the ones we agree with on 98 percent of topics.\n", - "\n", - "If someone you know, like, and trust believes a radical idea, you are more likely to give it merit, weight, or consideration. You already agree with them in most areas of life. Maybe you should change your mind on this one too. But if someone wildly different than you proposes the same radical idea, well, it’s easy to dismiss them as a crackpot.\n", - "\n", - "One way to visualize this distinction is by mapping beliefs on a spectrum. If you divide this spectrum into 10 units and you find yourself at Position 7, then there is little sense in trying to convince someone at Position 1. The gap is too wide. When you’re at Position 7, your time is better spent connecting with people who are at Positions 6 and 8, gradually pulling them in your direction.\n", - "\n", - "The most heated arguments often occur between people on opposite ends of the spectrum, but the most frequent learning occurs from people who are nearby. The closer you are to someone, the more likely it becomes that the one or two beliefs you don’t share will bleed over into your own mind and shape your thinking. The further away an idea is from your current position, the more likely you are to reject it outright.\n", - "\n", - "When it comes to changing people’s minds, it is very difficult to jump from one side to another. You can’t jump down the spectrum. You have to slide down it.\n", - "\n", - "Any idea that is sufficiently different from your current worldview will feel threatening. And the best place to ponder a threatening idea is in a non-threatening environment. As a result, books are often a better vehicle for transforming beliefs than conversations or debates.\n", - "\n", - "In conversation, people have to carefully consider their status and appearance. They want to save face and avoid looking stupid. When confronted with an uncomfortable set of facts, the tendency is often to double down on their current position rather than publicly admit to being wrong.\n", - "\n", - "Books resolve this tension. With a book, the conversation takes place inside someone’s head and without the risk of being judged by others. It’s easier to be open-minded when you aren’t feeling defensive.\n", - "\n", - "Arguments are like a full frontal attack on a person’s identity. Reading a book is like slipping the seed of an idea into a person’s brain and letting it grow on their own terms. There’s enough wrestling going on in someone’s head when they are overcoming a pre-existing belief. They don’t need to wrestle with you too.\n", - "\n", - "Why False Ideas Persist\n", - "There is another reason bad ideas continue to live on, which is that people continue to talk about them.\n", - "\n", - "Silence is death for any idea. An idea that is never spoken or written down dies with the person who conceived it. Ideas can only be remembered when they are repeated. They can only be believed when they are repeated.\n", - "\n", - "I have already pointed out that people repeat ideas to signal they are part of the same social group. But here’s a crucial point most people miss:\n", - "\n", - "People also repeat bad ideas when they complain about them. Before you can criticize an idea, you have to reference that idea. You end up repeating the ideas you’re hoping people will forget—but, of course, people can’t forget them because you keep talking about them. The more you repeat a bad idea, the more likely people are to believe it.\n", - "\n", - "Let’s call this phenomenon Clear’s Law of Recurrence: The number of people who believe an idea is directly proportional to the number of times it has been repeated during the last year—even if the idea is false.\n", - "\n", - "Each time you attack a bad idea, you are feeding the very monster you are trying to destroy. As one Twitter employee wrote, “Every time you retweet or quote tweet someone you’re angry with, it helps them. It disseminates their BS. Hell for the ideas you deplore is silence. Have the discipline to give it to them.”\n", - "\n", - "Your time is better spent championing good ideas than tearing down bad ones. Don’t waste time explaining why bad ideas are bad. You are simply fanning the flame of ignorance and stupidity.\n", - "\n", - "The best thing that can happen to a bad idea is that it is forgotten. The best thing that can happen to a good idea is that it is shared. It makes me think of Tyler Cowen’s quote, “Spend as little time as possible talking about how other people are wrong.”\n", - "\n", - "Feed the good ideas and let bad ideas die of starvation.\n", + "But for as long as we choose to share our lives with pets, the bar should be the same as for any relationship we value: being prepared to make sacrifices for their wellbeing, prioritising quality time and care, and loving them as they are – not for how they reflect on us, or how we’d like them to be.\n", "\n", "\n", "\"\"\"" @@ -128,7 +83,86 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 144, + "id": "50d5afda-418f-436b-b467-004863193d4a", + "metadata": {}, + "outputs": [], + "source": [ + "input_article_two = \"\"\"Lee Parkin had been the proud owner of his terrier-spaniel cross Izzy for nearly 10 years when he stepped out for what would be his last walk with his beloved pet.\n", + "\n", + "He was walking Izzy near his home in Doncaster when an XL bully pounced on her, mounting a 20-minute attack and ultimately killing the dog in front of Parkin, who desperately intervened in vain.\n", + "\n", + "“It was such a nice day,” he said. “We were walking a normal field where I go, and I saw this dog loose. It appeared wild by its demeanour.”\n", + "\n", + "Parkin, 50, took his dog through a gate but found himself cornered. The dog approached and started circling them. And then, he says, “it just grabbed her”.\n", + "\n", + "“I’ve never encountered a bigger, stronger dog before in my life,” he says. “I’ve dealt with dogs attacking another dog before.”\n", + "\n", + "Lee Parkin and his dog Izzy\n", + "Lee Parkin and his dog Izzy. Photograph: Lee Parkin\n", + "Parkin did his best to fight the dog off. “I smashed both hands against it, I twisted its balls, I kicked it in its back end. It did nothing whatsoever. I just shouted for help.”\n", + "\n", + "At first there were no other people around, but “all of a sudden” there were about three other men, possibly including the owner, attempting to remove the animal.\n", + "\n", + "A passerby gave him a lift to the vet but Izzy was “bleeding so profusely” he could hear her choking on her own blood. Her bones had been crushed.\n", + "\n", + "The owners were handed a caution and the dog remains alive and living nearby.\n", + "\n", + "“It was dangerously out of control,” Parkin says of the XL bully. “I’ve been brought up with dogs all my life. There’s no place for this type of dog in society.”\n", + "\n", + "He welcomes the incoming ban on XL bullies but says he does not think it is enough and it will not work.\n", + "\n", + "He believes the majority of XL bully owners will not be fazed by the ban and will keep their dogs and ignore the new law and regulations.\n", + "\n", + "“The only effective thing that I’ve seen the police doing is turning up and shooting these dogs, which is what I think they should be doing,” Parkin adds.\n", + "\n", + "He was left with significant mental impacts from the attack and was subsequently diagnosed with post-traumatic stress disorder. He received counselling but still struggles with walking dogs, and often rises very early in the morning to avoid other owners. He also carries a dog spray.\n", + "\n", + "Marie Hay’s siberian husky, Naevia, survived a savage attack on the beach in Redcar on the North Yorkshire coast by two XL bullies – but has been left with life-changing injuries. Hay, like Parkin, has also been left with mental scars.\n", + "\n", + "The owner of the dog that attacked seven-year-old Naevia is facing a criminal trial next year.\n", + "\n", + "“We must have only been three minutes and the guy pulls up and basically he’s just got his dogs out of the car. They run down to the bottom of the beach and one starts to run towards Naevia.\n", + "\n", + "“The owner turned to me and said: ‘They’re friendly, don’t worry,’ because I must have pulled a face at the size of the dog.\n", + "\n", + "skip past newsletter promotion\n", + "Sign up to First Edition\n", + "\n", + "Free daily newsletter\n", + "Our morning email breaks down the key stories of the day, telling you what’s happening and why it matters\n", + "\n", + "Enter your email address\n", + "Sign up\n", + "Privacy Notice: Newsletters may contain info about charities, online ads, and content funded by outside parties. For more information see our Privacy Policy. We use Google reCaptcha to protect our website and the Google Privacy Policy and Terms of Service apply.\n", + "after newsletter promotion\n", + "“But then the first one jumped on Naevia’s chest and just started tearing into her.\n", + "\n", + "“So she was screaming, screaming like a baby. And then the other one just came out of nowhere. The attack lasted about 12 minutes.”\n", + "\n", + "An American bully XL with cropped ears. The practice is illegal in England and Wales, but it is still carried out by unscrupulous owners.\n", + "Perfect pets or dangerous dogs? The sudden, surprising rise of American bully XLs\n", + "Read more\n", + "Hay said several people attempted to remove the dogs but were initially unsuccessful. They attempted to lift the dogs by the legs and her 20-year-old daughter was bitten, as were other people who intervened.\n", + "\n", + "The owner eventually placed a harness on one of them and put it in the car, while Hay had to walk the other dog back to the car on a lead.\n", + "\n", + "Naevia lost 83% of her blood. “She was bleeding to death on the beach … she had hundreds of bite marks all over, she had an incision that ripped her chest open.\n", + "\n", + "“She had to have between eight and 10 operations. She’s now in kidney failure because of the stress that it caused on her kidneys. She had to have two blood transfusions.”\n", + "\n", + "Hay said the vet bills were more than £30,000, which she has been able to cover through donations on a fundraising website.\n", + "\n", + "Like Parkin, Hay struggles to go out for walks now, due to the stress caused by the incident.\n", + "\n", + "“I carry a full kit that I’ve made myself, it’s got a rape alarm, a couple of extra dog leads … I’m constantly in fear.”\n", + "\n", + "Hay says she is “100%” supportive of the new ban. She says she accepts that a dog’s behaviour is partly down to the owners but is confident the breed plays a part too.\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 145, "id": "e328a903-d084-4d07-9b95-0a9196d7f719", "metadata": {}, "outputs": [], @@ -316,7 +350,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 146, "id": "af1b6a25-b37d-4861-82ae-fd74a7c69bc8", "metadata": {}, "outputs": [], @@ -336,7 +370,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 147, "id": "cf1965e3-e870-49a7-8ae5-fa4371e1c8f5", "metadata": {}, "outputs": [], @@ -371,7 +405,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 148, "id": "fad0c4b0-cd61-4c3c-9964-47f019278060", "metadata": {}, "outputs": [ @@ -379,18 +413,18 @@ "name": "stdout", "output_type": "stream", "text": [ - "{\"name\":\"Natural Language Text\",\"cognitive_subgroups\":[{\"id\":1,\"name\":\"Articles, essays, and reports\",\"data_type\":\"TEXT\"}]}\n" + "{\"name\":\"Natural Language Text\",\"cognitive_subgroups\":[{\"id\":1,\"name\":\"News stories and blog posts\",\"data_type\":\"TEXT\"}]}\n" ] } ], "source": [ - "required_layers = classify_input(input = input)\n", - "print(required_layers.json())" + "required_layers_one = classify_input(input = input_article_one)\n", + "print(required_layers_one.json())" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 149, "id": "06b483bf-2fa0-414f-8253-27ffe9a2881c", "metadata": {}, "outputs": [ @@ -398,78 +432,60 @@ "name": "stdout", "output_type": "stream", "text": [ - "{'name': 'Natural Language Text', 'cognitive_subgroups': [{'id': 1, 'name': 'Articles, essays, and reports', 'data_type': 'TEXT'}]}\n" + "{\"name\":\"Natural Language Text\",\"cognitive_subgroups\":[{\"id\":1,\"name\":\"News stories and blog posts\",\"data_type\":\"TEXT\"},{\"id\":2,\"name\":\"Personal narratives and stories\",\"data_type\":\"TEXT\"}]}\n" ] } ], "source": [ - "print(required_layers.dict())" + "required_layers_two = classify_input(input = input_article_two)\n", + "print(required_layers_two.json())" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "id": "35461aff-fd80-4eb2-94b2-66c742db8e55", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Hello\n" - ] - } - ], - "source": [ - "print(\"Hello\")" - ] + "outputs": [], + "source": [] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "id": "57e227bc-363d-437f-9430-c5d14aff6a31", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "TEXT\n" - ] - } - ], - "source": [ - "print(required_layers.dict()['cognitive_subgroups'][0]['data_type'])" - ] + "outputs": [], + "source": [] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 84, "id": "4112063c-e94c-4876-965e-1785e0682329", "metadata": {}, "outputs": [], "source": [ "\n", - "\n", - "system_prompt = f\"\"\"\n", - "You are tasked with analyzing a {required_layers.dict()['cognitive_subgroups'][0]['data_type']} files, especially in a multilayer network context for tasks such as analysis, categorization, and feature extraction, various layers can be incorporated to capture the depth and breadth of information contained within the {required_layers.dict()['cognitive_subgroups'][0]['data_type']} \n", - "These layers can help in understanding the content, context, and characteristics of the {required_layers.dict()['cognitive_subgroups'][0]['data_type']}\n", - "Your objective is to extract meaningful layers of information that will contribute to constructing a detailed multilayer network or knowledge graph.\n", - "Approach this task by considering the unique characteristics and inherent properties of the data at hand.\n", - "VERY IMPORTANT: The context you are working in is {required_layers.dict()['name']} and specific domain you are extracting data on is {required_layers.dict()['cognitive_subgroups'][0]['name']}\n", - "\n", - "Guidelines for Layer Extraction:\n", - "\n", - "Take into account: The content type that in this case is: {required_layers.dict()['cognitive_subgroups'][0]['name']} should play a major role in how you decompose into layers.\n", - "\n", - "Based on your analysis, define and describe the layers you've identified, explaining their relevance and contribution to understanding the dataset. Your independent identification of layers will enable a nuanced and multifaceted representation of the data, enhancing applications in knowledge discovery, content analysis, and information retrieval.\n", - "\n", - ".\"\"\"" + "def system_prompt_temp(required_layers):\n", + " system_prompt = f\"\"\"\n", + " You are tasked with analyzing a {required_layers.dict()['cognitive_subgroups'][0]['data_type']} files, especially in a multilayer network context for tasks such as analysis, categorization, and feature extraction, various layers can be incorporated to capture the depth and breadth of information contained within the {required_layers.dict()['cognitive_subgroups'][0]['data_type']} \n", + " These layers can help in understanding the content, context, and characteristics of the {required_layers.dict()['cognitive_subgroups'][0]['data_type']}\n", + " Your objective is to extract meaningful layers of information that will contribute to constructing a detailed multilayer network or knowledge graph.\n", + " Approach this task by considering the unique characteristics and inherent properties of the data at hand.\n", + " VERY IMPORTANT: The context you are working in is {required_layers.dict()['name']} and specific domain you are extracting data on is {required_layers.dict()['cognitive_subgroups'][0]['name']}\n", + " \n", + " Guidelines for Layer Extraction:\n", + " \n", + " Take into account: The content type that in this case is: {required_layers.dict()['cognitive_subgroups'][0]['name']} should play a major role in how you decompose into layers.\n", + " \n", + " Based on your analysis, define and describe the layers you've identified, explaining their relevance and contribution to understanding the dataset. Your independent identification of layers will enable a nuanced and multifaceted representation of the data, enhancing applications in knowledge discovery, content analysis, and information retrieval.\n", + " \n", + " .\"\"\"\n", + " return system_prompt" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 85, "id": "7c5baaed-5447-425b-bc9c-03d071d66187", "metadata": {}, "outputs": [], @@ -489,12 +505,12 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 86, "id": "ee9c8dad-00ee-48ec-bcb5-8b9c74f91141", "metadata": {}, "outputs": [], "source": [ - "def determine_layers(input) -> CognitiveLayer:\n", + "def determine_layers(input, required_layers) -> CognitiveLayer:\n", " \"\"\"Classify input\"\"\"\n", " model = \"gpt-4-1106-preview\"\n", " user_prompt = f\"Use the given format to extract information from the following input: {input}.\"\n", @@ -509,7 +525,7 @@ " },\n", " {\n", " \"role\": \"system\",\n", - " \"content\": system_prompt,\n", + " \"content\": system_prompt_temp(required_layers),\n", " },\n", " ],\n", " response_model=CognitiveLayer,\n", @@ -519,17 +535,17 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 87, "id": "f4f59ef6-5cdd-478c-a96e-d2de2cc0e04f", "metadata": {}, "outputs": [], "source": [ - "cognitive_layers = determine_layers(input=input)" + "cognitive_layers_one = determine_layers(input=input_article_one, required_layers= required_layers_one)" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 88, "id": "15631e68-61dc-4955-853f-52bf0cb93fbb", "metadata": {}, "outputs": [ @@ -537,17 +553,65 @@ "name": "stdout", "output_type": "stream", "text": [ - "category_name='Natural Language Text' cognitive_layers=[CognitiveLayerSubgroup(id=1, name='Thematic Content Layer', description='This layer captures the central themes, topics, or subject matter of the articles, essays, and reports. It involves analyzing the text for key messages, main arguments, and overarching ideologies, which aids in understanding the focus and direction of the content.'), CognitiveLayerSubgroup(id=2, name='Structural Composition Layer', description=\"This layer encompasses the organization and structure of the text, including headings, subheadings, paragraphs, and overall layout. Understanding how the text is organized can reveal the author's approach to presenting information and building their argument.\"), CognitiveLayerSubgroup(id=3, name='Linguistic Style Layer', description=\"This layer analyzes the linguistic choices and writing styles used in the text, such as vocabulary, sentence construction, and use of literary devices. It provides insights into the author's tone, level of formality, and persuasive techniques.\"), CognitiveLayerSubgroup(id=4, name='Contextual Relevance Layer', description='This layer examines the text in relation to its historical, cultural, or situational context. It considers external factors that may influence the interpretation of the text, including the time period it was written in and its cultural significance.'), CognitiveLayerSubgroup(id=5, name='Semantic Connectivity Layer', description='This layer focuses on the relationships between concepts, ideas, and entities within the text. It involves identifying connections, references, and allusions that contribute to the deeper meaning and understanding of the content.'), CognitiveLayerSubgroup(id=6, name='Rhetorical Strategies Layer', description='This layer scrutinizes the rhetorical techniques and strategies employed by the author to persuade the audience. It looks at the use of ethos, pathos, and logos, and how these contribute to the effectiveness of the argument.'), CognitiveLayerSubgroup(id=7, name='Audience Engagement Layer', description='This layer considers the projected audience and the potential impact or response the text could elicit. It includes analyzing how the text addresses the reader, anticipates their questions or concerns, and engages them in the discourse.'), CognitiveLayerSubgroup(id=8, name='Narrative Flow Layer', description='This layer assesses the flow and progression of ideas throughout the text. It looks at how smoothly the text transitions from one point to another, the pacing of information delivery, and the narrative techniques used to maintain interest.'), CognitiveLayerSubgroup(id=9, name='Cognitive Dissonance Layer', description=\"This layer identifies instances where the text presents ideas or arguments that may challenge the reader's pre-existing beliefs or assumptions. It examines how the text handles possible contradictions and the resolution of conflicting viewpoints.\"), CognitiveLayerSubgroup(id=10, name='Social Influence Layer', description=\"This layer explores the social implications of the text, including the influence on the reader's beliefs, behaviors, and social dynamics. It also considers the potential for the text to reflect or shape societal norms and values.\")]\n" + "category_name='Exploring British Attitudes Toward Pets' cognitive_layers=[CognitiveLayerSubgroup(id=1, name='Cultural Practices', description='Exploration of British cultural practices regarding pets, highlighting the affection and accommodations made for dogs in public spaces, and the sociocultural emphasis on pet ownership.'), CognitiveLayerSubgroup(id=2, name='Behavioral Shifts', description='Identification and analysis of changing behaviors and attitudes toward pets in Britain, particularly during and after the COVID-19 pandemic.'), CognitiveLayerSubgroup(id=3, name='Animal Welfare Concerns', description='Insights into the repercussions of current trends in pet ownership on animal welfare, including the impact of designer breeds and the commodification of pets.'), CognitiveLayerSubgroup(id=4, name='Human-Pet Dynamics', description='Discussion of the evolving relationship between humans and pets, focusing on the anthropomorphization of animals, and the appearance versus functionality of pets.'), CognitiveLayerSubgroup(id=5, name='Economic Impact', description='Assessment of the financial implications of pet ownership in Britain, with particular attention to consumerism and the booming pet industry.'), CognitiveLayerSubgroup(id=6, name='Social Commentary', description=\"A layer that encapsulates the author's critique of society's indulgence in pet-related consumerism, often at the expense of the pets' natural needs and well-being.\"), CognitiveLayerSubgroup(id=7, name='Ethical Considerations', description='Critical examination of the moral aspects of pet ownership practices, questioning the ethics behind breeding, commodification, and the treatment of pets as surrogate children.'), CognitiveLayerSubgroup(id=8, name='Comparative Analysis', description='Comparing British attitudes and practices toward pets with those of other countries, drawing contrasts and identifying unique characteristics.'), CognitiveLayerSubgroup(id=9, name='Public Policy Implications', description='Implications for public policy and the need for regulation or intervention in pet-related matters, considering the rising issues stemming from current societal attitudes.')]\n" ] } ], "source": [ - "print(cognitive_layers)" + "print(cognitive_layers_one)" ] }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 89, + "id": "36ef0bd6-9510-4265-a736-5e10ef1799d0", + "metadata": {}, + "outputs": [], + "source": [ + "cognitive_layers_two = determine_layers(input=input_article_two, required_layers=required_layers_two)" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "id": "8d377859-ed21-4408-ae72-79e8bc9a5309", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "category_name='News stories and blog posts' cognitive_layers=[CognitiveLayerSubgroup(id=1, name='Topical Context', description='This layer encapsulates the main subject matter of the text, which involves animal attacks, specifically dog attacks, and their legal and emotional repercussions.'), CognitiveLayerSubgroup(id=2, name='Geographical Context', description='The layer that locates the events mentioned in the text geographically, including specific places like Doncaster, Redcar, and the North Yorkshire coast in England.'), CognitiveLayerSubgroup(id=3, name='Temporal Context', description='This layer provides temporal references pertinent to the stories related to the individual attacks and subsequent events, such as the upcoming criminal trial or the historical context of the pet ownership.'), CognitiveLayerSubgroup(id=4, name='Personal Testimony', description='A layer of direct quotes and personal experiences as recounted by the individuals involved in the incidents, emphasizing the personal and emotional aspect of the news stories.'), CognitiveLayerSubgroup(id=5, name='Legal and Policy Framework', description='This layer outlines the impact of current and upcoming legistlations on dog ownership and the societal responses to breed-specific bans or laws.'), CognitiveLayerSubgroup(id=6, name='Socioemotional Impact', description='Compiling the reported psychological and emotional effects following the dog attacks, including mental health issues like PTSD and fear of walking dogs.'), CognitiveLayerSubgroup(id=7, name='Economic Impact', description='This layer includes the financial repercussions for the victims, such as veterinary bills and the costs associated with dog attacks.'), CognitiveLayerSubgroup(id=8, name='Community Reaction', description=\"A layer that includes reactions from the broader community, including the response to calls for assistance during attacks and the public's stance on breed-specific legislation.\"), CognitiveLayerSubgroup(id=9, name='Advocacy and Activism', description='This layer covers the efforts by individuals and groups to advocate for changes in law, public awareness of dangerous breeds, and community support through fundraising.'), CognitiveLayerSubgroup(id=10, name='Incident Details', description=\"A descriptive layer providing information on the specifics of the dog attacks, including the nature of the attacks, the dogs involved, and the owners' actions during and after the incidents.\")]\n" + ] + } + ], + "source": [ + "print(cognitive_layers_two)" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "41d06ecb-83b9-4284-8d88-6a3f710cb457", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Extracted Layer Names: ['Cultural Practices', 'Behavioral Shifts', 'Animal Welfare Concerns', 'Human-Pet Dynamics', 'Economic Impact', 'Social Commentary', 'Ethical Considerations', 'Comparative Analysis', 'Public Policy Implications']\n" + ] + } + ], + "source": [ + "cognitive_layers_one = [layer_subgroup.name for layer_subgroup in cognitive_layers_one.cognitive_layers]\n", + "\n", + "print(\"Extracted Layer Names:\", cognitive_layers_one)" + ] + }, + { + "cell_type": "code", + "execution_count": 92, "id": "1a287a2a-2fb5-4ad3-a69e-80ed2e2ffa5a", "metadata": {}, "outputs": [ @@ -555,38 +619,39 @@ "name": "stdout", "output_type": "stream", "text": [ - "Extracted Layer Names: ['Thematic Content Layer', 'Structural Composition Layer', 'Linguistic Style Layer', 'Contextual Relevance Layer', 'Semantic Connectivity Layer', 'Rhetorical Strategies Layer', 'Audience Engagement Layer', 'Narrative Flow Layer', 'Cognitive Dissonance Layer', 'Social Influence Layer']\n" + "Extracted Layer Names: ['Topical Context', 'Geographical Context', 'Temporal Context', 'Personal Testimony', 'Legal and Policy Framework', 'Socioemotional Impact', 'Economic Impact', 'Community Reaction', 'Advocacy and Activism', 'Incident Details']\n" ] } ], "source": [ - "layer_names = [layer_subgroup.name for layer_subgroup in cognitive_layers.cognitive_layers]\n", + "cognitive_layers_two = [layer_subgroup.name for layer_subgroup in cognitive_layers_two.cognitive_layers]\n", "\n", - "print(\"Extracted Layer Names:\", layer_names)" + "print(\"Extracted Layer Names:\", cognitive_layers_two)" ] }, { "cell_type": "code", - "execution_count": 40, + "execution_count": null, + "id": "609b1287-e0bf-42a5-856a-f2e0d859ea8b", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "001466ea-d805-444e-9d70-5505d83eb980", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, "id": "07014c19-e981-4150-afc0-78800062f6e0", "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "['Thematic Content Layer',\n", - " 'Structural Composition Layer',\n", - " 'Linguistic Style Layer']" - ] - }, - "execution_count": 40, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "layer_names[:3]\n" - ] + "outputs": [], + "source": [] }, { "cell_type": "code", @@ -598,7 +663,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 93, "id": "dbce6243-7501-42d1-b944-f80811ae903d", "metadata": {}, "outputs": [], @@ -644,7 +709,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 100, "id": "12a0d829-1387-4e32-84a1-1ad7b6edf0dc", "metadata": {}, "outputs": [], @@ -718,66 +783,79 @@ }, { "cell_type": "code", - "execution_count": 41, + "execution_count": 101, "id": "12bf38da-019d-4568-af21-21507c60f906", "metadata": {}, + "outputs": [], + "source": [ + "\n", + "def create_layer_graph(input, layer_list):\n", + " layer_graphs = []\n", + " \n", + " for layer in layer_list[:3]:\n", + " print(\"Layer processed is:\", str(layer))\n", + " layer_graph = generate_graph(input=input, layer= layer)\n", + " print(\"Layer graph is:\", str(layer_graph))\n", + " layer_graphs.append(layer_graph)\n", + " return layer_graphs\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "id": "4a19cc82-b892-47f3-99db-b70edccefda5", + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Layer processed is: Thematic Content Layer\n", - "Layer graph is: nodes=[Node(id=1, description='Economist who wrote about the preference of people to prove existing beliefs over changing their minds.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Believed that unformed minds can understand difficult subjects but biased minds cannot accept even simple things.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Humans prioritize social belonging over accuracy, causing them to hold beliefs for social advantage rather than truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='A powerful human need which often conflicts with the pursuit of truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='The conflict between social belonging and understanding the truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='Suggests that the brain holds beliefs that bring social alliances over strictly true beliefs.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Notes that the brain is rewarded for adopting certain beliefs regardless of their factual accuracy.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='A term used to describe beliefs that are incorrect but socially beneficial to hold.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='The idea that friendships and social ties have a greater impact on changing minds than factual evidence.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Advocates sharing meals with those of opposing views to promote tolerance and understanding.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='A conceptual framework that suggests engaging with people who hold similar but not identical beliefs to effectively change minds.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='Introduced the idea that minds are more likely to be changed by those we mostly agree with.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The concept that books, due to their private and introspective nature, can facilitate belief change better than public debates.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description='States that the prevalence of an idea is proportional to how much it is repeated, regardless of its truth.', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=15, description='Suggests spending as little time as possible discussing the wrongness of others to starve bad ideas and promote good ones.', category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description=\"Galbraith's quote reflects the logic of false beliefs.\", created_at=None, summarized=None), Edge(source=2, target=3, description=\"Tolstoy's quote supports the ideas about the persistence of false beliefs.\", created_at=None, summarized=None), Edge(source=4, target=3, description='The need for social belonging is central to the logic of false beliefs.', created_at=None, summarized=None), Edge(source=5, target=3, description='Tribalism highlights the conflict in the logic of false beliefs.', created_at=None, summarized=None), Edge(source=6, target=3, description='Pinker provides psychological insight into the logic of false beliefs.', created_at=None, summarized=None), Edge(source=7, target=3, description=\"Simler's views add to the understanding of the logic of false beliefs.\", created_at=None, summarized=None), Edge(source=8, target=3, description='The term explains a key aspect of the logic of false beliefs.', created_at=None, summarized=None), Edge(source=9, target=3, description='This idea illustrates an exception within the logic of false beliefs.', created_at=None, summarized=None), Edge(source=10, target=9, description='de Botton suggests a practical application of friendship over facts.', created_at=None, summarized=None), Edge(source=11, target=9, description=\"The spectrum of beliefs framework supports the 'friendship over facts' idea.\", created_at=None, summarized=None), Edge(source=12, target=11, description=\"Casnocha's idea is at the core of the spectrum of beliefs.\", created_at=None, summarized=None), Edge(source=13, target=11, description='Books serve as an effective way to change beliefs within the spectrum of beliefs.', created_at=None, summarized=None), Edge(source=14, target=3, description=\"Clear's Law outlines a phenomenon relevant to the persistence of false beliefs.\", created_at=None, summarized=None), Edge(source=15, target=14, description=\"Cowen's quote complements Clear's Law of Recurrence with a strategy to diminish bad ideas.\", created_at=None, summarized=None)]\n", - "Layer processed is: Structural Composition Layer\n", - "Layer graph is: nodes=[Node(id=1, description='The economist J.K. Galbraith', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='The writer Leo Tolstoy', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='The phenomenon of belief persistence despite contradicting facts', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='The psychology behind preferring social belonging over factual correctness', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description=\"The book 'Atomic Habits' discussing humans as herd animals and the need to fit in\", category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='The Harvard psychologist Steven Pinker', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='The concept that rewards from peers encourage belief adoption', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Kevin Simler, a writer who discussed the influence of rewards on belief adoption', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='The idea that friendship and social connections have a stronger impact on belief change than facts', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The British philosopher Alain de Botton', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='The concept of a spectrum of beliefs and the likelihood of changing minds', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='Ben Casnocha, a person who introduced the idea of the spectrum of beliefs', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The notion that books are more effective than conversations for belief change', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description='The persistence of false ideas due to their continued discussion', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=15, description=\"Clear's Law of Recurrence, posited by James Clear\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=16, description=\"Tyler Cowen, an economist who comments on the nature of discussion about others' wrongs\", category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description=\"discussed the concept of proof over changing one's mind\", created_at=None, summarized=None), Edge(source=2, target=3, description='commented on the explanation of subjects relative to preconceived notions', created_at=None, summarized=None), Edge(source=5, target=4, description='introduces the concept of human sociability and tribe belonging', created_at=None, summarized=None), Edge(source=6, target=4, description='explains the social aspect of belief adoption', created_at=None, summarized=None), Edge(source=8, target=7, description='discusses the reward system in brain related to beliefs', created_at=None, summarized=None), Edge(source=10, target=9, description='suggests sharing meals to bridge differences', created_at=None, summarized=None), Edge(source=12, target=11, description='introduced idea to author James Clear about spectrum of beliefs', created_at=None, summarized=None), Edge(source=16, target=14, description=\"advises less focus on discussing others' wrongs\", created_at=None, summarized=None)]\n", - "Layer processed is: Linguistic Style Layer\n", - "Layer graph is: nodes=[Node(id=1, description='The economist who wrote about the challenges people face when having to change their minds.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='The writer who suggested that unformed ideas can be explained to anyone regardless of intelligence', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Have a desire to belong and maintain social connections, sometimes at the cost of accepting false beliefs.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Harvard psychologist who spoke about the social function of beliefs in relation to acceptance in society.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description=\"Wrote about the brain's willingness to adopt beliefs for rewards, whether pragmatic or social.\", category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description=\"Often less influential than social connections in changing a person's mind or beliefs.\", category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='A more effective way to change minds than presenting facts, as it involves social integration.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='British philosopher who suggests that sharing meals can promote tolerance and reduce prejudice.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='A concept where beliefs are visualized on a spectrum, showing that nearby beliefs are more easily influenced.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Described as a better medium for transforming beliefs as they allow ideas to be considered without social pressures.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='The idea that the more an idea is repeated, the more it is believed, regardless of its veracity.', category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description=\"Advocates spending minimal time on highlighting others' mistakes and instead focusing on positive ideas.\", category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='A book that discusses human inclinations to belong, bond, and be accepted by others.', category='entity', memory_type='episodic', created_at=None, summarized=None), Node(id=14, description=\"Author of 'Atomic Habits', discusses the logic of false beliefs and the importance of social connections over facts.\", category='person', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description='discussed challenges faced by humans when changing their minds', created_at=None, summarized=None), Edge(source=2, target=3, description='suggested that preconceived notions hinder understanding simple concepts', created_at=None, summarized=None), Edge(source=3, target=4, description='Steven Pinker provides insight on human beliefs and their social function', created_at=None, summarized=None), Edge(source=3, target=5, description=\"Kevin Simler wrote about human brain's response towards adopting beliefs for various rewards\", created_at=None, summarized=None), Edge(source=3, target=6, description='tend to prioritize social connections over facts when changing beliefs', created_at=None, summarized=None), Edge(source=6, target=7, description=\"less influential than friendship in changing people's beliefs\", created_at=None, summarized=None), Edge(source=7, target=8, description='Alain de Botton suggests, through friendship, shared meals can help bridge differences', created_at=None, summarized=None), Edge(source=3, target=9, description='are more inclined to change their beliefs when influenced by those they are socially or ideologically close to', created_at=None, summarized=None), Edge(source=3, target=10, description='find books to be a non-threatening environment for considering new ideas', created_at=None, summarized=None), Edge(source=3, target=11, description=\"tend to believe ideas that are repeatedly mentioned, as described in Clear's Law of Recurrence\", created_at=None, summarized=None), Edge(source=12, target=11, description=\"emphasizes not to spend time on bad ideas, aligning with Clear's Law of Recurrence\", created_at=None, summarized=None), Edge(source=14, target=13, description='wrote the book discussing human social inclinations', created_at=None, summarized=None)]\n" + "Layer processed is: Cultural Practices\n", + "Layer graph is: nodes=[Node(id=1, description='In British culture, keeping pets is an integral way of life and an outlet for emotions, with a particular fondness for dogs', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Dogs serve as an acceptable outlet for typically controlled British emotions, facilitating affection and sociability', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description=\"Public spaces in the UK are accommodating of dogs, with signs such as 'Dogs welcome, people tolerated' and treats offered in establishments\", category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Dogs act as social catalysts, making it more acceptable to converse with strangers in parks rather than more formal settings like trains', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='The COVID-19 pandemic caused a surge in pet ownership in the UK, with the dog population rising from 9 million to 13 million between 2019 and 2022', category='Events', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description='England and Wales saw a significant increase in dog attacks between 2018 and 2022, aligning with a rise in the number of dogs and changes in ownership dynamics', category='Events', memory_type='episodic', created_at=None, summarized=None), Node(id=7, description=\"There is a concerning trend of treating living beings, particularly popular 'designer' dog breeds, as commodities rather than sentient beings\", category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description=\"The cultural shift in Britain towards treating pets as substitutes for children, favoring aesthetics over the animal's natural function and well-being\", category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Consumerism in pet care is escalating in the UK with a significant increase in unnecessary spending on pets, including trends and luxury items', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Pets are often indulged with unnecessary treats and services that do not align with their genuine needs or desires, indicative of human emotional solace through consumption', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='Modern lifestyles with busy schedules are causing dogs to live monotonous lives, leading to behavioral problems that are often placated by trivial indulgences', category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description=\"Proper pet care advocates for prioritizing pets' quality of life and well-being over anthropomorphic indulgences and misguided expressions of affection\", category='Cultural Practices', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='Cultural norm facilitates', created_at=None, summarized=None), Edge(source=2, target=4, description='Function as', created_at=None, summarized=None), Edge(source=1, target=3, description='Manifested in', created_at=None, summarized=None), Edge(source=5, target=6, description='Correlated with', created_at=None, summarized=None), Edge(source=1, target=7, description='Increasing trend in', created_at=None, summarized=None), Edge(source=7, target=8, description='Underpinned by', created_at=None, summarized=None), Edge(source=1, target=9, description='Leads to', created_at=None, summarized=None), Edge(source=10, target=9, description='Driven by', created_at=None, summarized=None), Edge(source=11, target=12, description='Counteracted by prioritizing', created_at=None, summarized=None)]\n", + "Layer processed is: Behavioral Shifts\n", + "Layer graph is: nodes=[Node(id=1, description=\"Kate Fox, an anthropologist who described Britons' affection towards pets as a way of life in her book 'Watching the English'\", category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Dogs serve as an outlet for emotions and social engagement for Britons', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='In the UK, dogs are accepted and encouraged in various public spaces like transport, pubs, and shops', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Despite societal changes, the passion for animals in the UK has been consistent', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='The COVID-19 pandemic led to a surge in pet ownership, increasing the number of dogs in the UK', category='event', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description=\"'A dog is for life, not just for Christmas' is a slogan by Dogs Trust charity to advocate responsible pet ownership\", category='entity', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='The number of dog attacks in England and Wales saw a significant rise', category='event', memory_type='episodic', created_at=None, summarized=None), Node(id=8, description=\"Certain dog breeds are increasingly perceived as commodities, particularly 'designer' breeds\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Consumer spending on pets in the UK has increased, extending to non-essential items and services', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Many dogs experience lack of socialization and extended periods of isolation due to modern human schedules', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='Ensuring the well-being of pets should be a priority, demanding sacrifices and quality care from the owners', category='concept', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='Kate Fox highlighted the role of dogs as emotional outlets in British culture', created_at=None, summarized=None), Edge(source=2, target=3, description='The role of dogs in emotional support correlates with their acceptance in public spaces', created_at=None, summarized=None), Edge(source=4, target=5, description='The consistent passion for animals in the UK manifested as a pet boom during the pandemic', created_at=None, summarized=None), Edge(source=5, target=6, description='The increase in pet ownership during the pandemic prompted reminders of lifelong commitment to pets', created_at=None, summarized=None), Edge(source=5, target=7, description='The pet boom during the pandemic coincided with an increase in dog attacks', created_at=None, summarized=None), Edge(source=7, target=8, description='The rise in dog attacks is associated with viewing dogs as commodities rather than companions', created_at=None, summarized=None), Edge(source=8, target=9, description='The commodification of dogs connects to excessive consumer spending on pets', created_at=None, summarized=None), Edge(source=9, target=10, description='Despite consumer indulgence, many pets face isolation and lack of socialization', created_at=None, summarized=None), Edge(source=10, target=11, description='The isolation of pets challenges the principle of prioritizing their well-being', created_at=None, summarized=None)]\n", + "Layer processed is: Animal Welfare Concerns\n", + "Layer graph is: nodes=[Node(id=1, description='Britons have a significant and emotional connection with animals, particularly dogs.', category='culturalTrait', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Kate Fox, an anthropologist who wrote about the English way of life with pets.', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Number of pet dogs in the UK increased from about nine million to 13 million between 2019 and 2022.', category='statistic', memory_type='episodic', created_at=None, summarized=None), Node(id=4, description='In the UK, dogs are often encouraged to accompany their owners in public places such as pubs and shops.', category='socialNorm', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='The number of dog attacks recorded by police in England and Wales rose by more than a third between 2018 and 2022.', category='statistic', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description='Pets, particularly dogs, are increasingly being treated as commodities rather than sentient beings.', category='issue', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Popularity of designer breeds like French bulldogs and pugs, which often suffer due to their physical features.', category='trend', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Controversial American XL bully dogs are sought after as a signifier of masculinity.', category='trend', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Pets are increasingly being treated as substitutes for children, altering the traditional British perspective of valuing dogs for their natural qualities.', category='culturalShift', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='Spending on pets in the UK has more than doubled in the past decade, reaching nearly £10bn in the previous year.', category='economicData', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='Rise in dog-friendly establishments like restaurants, theaters, and even churches, catering to dog owners more than the dogs themselves.', category='socialTrend', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description=\"Many dogs suffer from daily deprivation and lack engagement due to their owner's busy schedules.\", category='welfareIssue', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='Behavioral issues in dogs often stem from insufficient exercise, supervision, and consistent routines.', category='welfareIssue', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description=\"Kate Fox wrote about Britons' significant relationship with animals.\", created_at=None, summarized=None), Edge(source=1, target=3, description='The connection Britons have with animals is illustrated by the increase in pet dogs.', created_at=None, summarized=None), Edge(source=1, target=4, description=\"Britons' affection for animals is evidenced by the encouragement of dogs in public places.\", created_at=None, summarized=None), Edge(source=3, target=5, description='The increase in pet dogs has correlated with a rise in dog attacks.', created_at=None, summarized=None), Edge(source=1, target=6, description='The change in how Britons relate to animals can be seen in the commodification of dogs.', created_at=None, summarized=None), Edge(source=6, target=7, description='Treatment of dogs as commodities is partly due to the popularity of designer breeds.', created_at=None, summarized=None), Edge(source=6, target=8, description='The view of dogs as commodities extends to the demand for breeds like American XL bully dogs.', created_at=None, summarized=None), Edge(source=7, target=8, description='American XL bully dogs are part of the trend for designer dog breeds.', created_at=None, summarized=None), Edge(source=1, target=9, description='Pets are increasingly seen as child substitutes, showing a shift in the traditional British view of domestic animals.', created_at=None, summarized=None), Edge(source=10, target=11, description='The rise in spending on pets coincides with the increase of dog-friendly establishments.', created_at=None, summarized=None), Edge(source=12, target=13, description='Daily deprivation experienced by many dogs contributes to their behavioral issues.', created_at=None, summarized=None)]\n", + "Layer processed is: Topical Context\n", + "Layer graph is: nodes=[Node(id=1, description='Britons have a strong affection for animals, especially dogs, which serve as outlets for restrained emotions.', category='Cultural Phenomenon', memory_type='Semantic', created_at=None, summarized=None), Node(id=2, description='In the UK, dogs are allowed and encouraged on public transport, a policy contrasting with Australia or New Zealand.', category='Policy', memory_type='Semantic', created_at=None, summarized=None), Node(id=3, description='The number of pet dogs in the UK increased from about 9 million to 13 million between 2019 and 2022.', category='Statistic', memory_type='Semantic', created_at=None, summarized=None), Node(id=4, description='Dogs are increasingly treated as commodities, valued for aesthetic appeal rather than functionality.', category='Cultural Shift', memory_type='Semantic', created_at=None, summarized=None), Node(id=5, description=\"There's a cultural shift in treating pets as child substitutes, characterized by indulgence in unnecessary products and services.\", category='Cultural Shift', memory_type='Semantic', created_at=None, summarized=None), Node(id=6, description='Pet expenditure in the UK has more than doubled in the past decade, reaching nearly £10 billion.', category='Economic Statistic', memory_type='Semantic', created_at=None, summarized=None), Node(id=7, description='Rise in dog behavioral problems due to lack of exercise, supervision, and consistent routines.', category='Animal Welfare Issue', memory_type='Semantic', created_at=None, summarized=None), Node(id=8, description='Pet wellbeing should be prioritized with quality care, acknowledging their nature, and fostering good human-animal relationships.', category='Animal Welfare Principle', memory_type='Semantic', created_at=None, summarized=None), Node(id=9, description=\"There's been more than a third increase in dog attacks recorded by police in England and Wales between 2018 and 2022.\", category='Crime Statistic', memory_type='Semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='Affection for animals in the UK reflects in their dog-friendly transportation policies.', created_at=None, summarized=None), Edge(source=3, target=4, description='The increase in pet dogs correlates with their growing treatment as commodities.', created_at=None, summarized=None), Edge(source=4, target=5, description='The commodification of pets relates to the broader cultural shift of treating them as child substitutes.', created_at=None, summarized=None), Edge(source=6, target=5, description='Rising pet expenditures reflect the trend of over-indulging pets in a manner similar to child substitutes.', created_at=None, summarized=None), Edge(source=7, target=9, description='Behavioral issues in dogs may be contributing to the rising number of dog attacks.', created_at=None, summarized=None), Edge(source=8, target=7, description='Prioritizing pet wellbeing can help address behavioral problems in pets.', created_at=None, summarized=None)]\n", + "Layer processed is: Geographical Context\n", + "Layer graph is: nodes=[Node(id=1, description='A country in Western Europe known for accommodating dogs and allowing them on public transport', category='geographicalLocation', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='A town in the United Kingdom, noted as one of the most deprived areas', category='geographicalLocation', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='A city in the United Kingdom where pet boutiques are a sign of gentrification', category='geographicalLocation', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='A country mentioned in comparison to the United Kingdom, where the approach to dogs on public transport differs', category='geographicalLocation', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='A country mentioned in comparison to the United Kingdom, where the approach to dogs on public transport differs', category='geographicalLocation', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='Great Yarmouth is a town within the United Kingdom', created_at=None, summarized=True), Edge(source=1, target=3, description='Norwich is a city within the United Kingdom', created_at=None, summarized=True)]\n", + "Layer processed is: Temporal Context\n", + "Layer graph is: nodes=[Node(id=1, description='Britons have a strong affection for animals, treating pet keeping not just as a leisure activity but as a way of life.', category='cultural trait', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='In Britain, dogs serve as an acceptable outlet for emotions and impulses such as affection and sociability.', category='behavior', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Dogs are permitted and encouraged on public transport and welcomed in many pubs and shops in the UK.', category='social norm', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description=\"The Dogs Trust charity created the slogan 'A dog is for life, not just for Christmas' in 1978.\", category='campaign', memory_type='episodic', created_at=1978.0, summarized=None), Node(id=5, description='The number of pet dogs in the UK increased from about nine million to 13 million between 2019 and 2022.', category='statistic', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='The number of dog attacks recorded by police in England and Wales rose by more than a third between 2018 and 2022.', category='statistic', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='During the COVID-19 pandemic, many people acquired dogs to make the most of time at home and in local parks.', category='event', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description=\"Living beings, especially certain dog breeds, are being treated as commodities, reflecting 'designer' trends.\", category='social issue', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description=\"The treatment of pets in Britain is shifting towards an 'American' approach, with pets increasingly substituting for children.\", category='cultural shift', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The trend of consumerism is extending to pets in the UK, with high expenditure on non-essential pet products and services.', category='trend', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description='The wellbeing of dogs is often compromised by modern lifestyles that restrict their social and physical needs.', category='welfare concern', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=7, target=5, description='The COVID-19 pandemic led to an increase in pet dog ownership in the UK.', created_at=None, summarized=None), Edge(source=8, target=10, description='The treatment of pets as commodities is associated with the consumerism trend of buying unnecessary pet products.', created_at=None, summarized=None), Edge(source=6, target=11, description='The rise in dog attacks may relate to the compromised wellbeing of dogs due to modern lifestyle constraints.', created_at=None, summarized=None)]\n" ] } ], "source": [ - "layer_graphs = []\n", - "\n", - "for layer in layer_names[:3]:\n", - " print(\"Layer processed is:\", str(layer))\n", - " layer_graph = generate_graph(input=input, layer= layer)\n", - " print(\"Layer graph is:\", str(layer_graph))\n", - " layer_graphs.append(layer_graph)\n", - "\n" + "layer_1_graph = create_layer_graph(input_article_one, cognitive_layers_one)\n" ] }, { "cell_type": "code", - "execution_count": 18, - "id": "4a19cc82-b892-47f3-99db-b70edccefda5", + "execution_count": 103, + "id": "15dc7863-0f4c-47ae-89ef-2656e8478249", "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "KnowledgeGraph(nodes=[Node(id=1, description='J.K. Galbraith', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=2, description='Leo Tolstoy', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=3, description='Steven Pinker', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='Kevin Simler', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='Alain de Botton', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=6, description='Abraham Lincoln', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=7, description='Ben Casnocha', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Tyler Cowen', category='person', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='Atomic Habits - Book', category='creativeWork', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='The logic and benefits of holding false beliefs', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=11, description=\"Facts don't change our minds, friendship does\", category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=12, description='The spectrum of beliefs and changeability', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=13, description='The persistence of false ideas through recurrence', category='concept', memory_type='semantic', created_at=None, summarized=None), Node(id=14, description=\"Clear's Law of Recurrence\", category='concept', memory_type='semantic', created_at=None, summarized=None)], edges=[Edge(source=1, target=10, description=\"J.K. Galbraith wrote about the human tendency to prove existing beliefs rather than change one's mind.\", created_at=None, summarized=None), Edge(source=2, target=10, description='Leo Tolstoy noted the difficulty of explaining things to someone who is certain in their pre-existing beliefs.', created_at=None, summarized=None), Edge(source=9, target=10, description='Atomic Habits discusses the importance of social belonging over accuracy of beliefs.', created_at=None, summarized=None), Edge(source=3, target=10, description='Steven Pinker suggests that beliefs may function to secure social allies over truth.', created_at=None, summarized=None), Edge(source=4, target=10, description=\"Kevin Simler described beliefs as anticipatory rewards, with little care for the reward's origin.\", created_at=None, summarized=None), Edge(source=5, target=11, description='Alain de Botton proposed sharing meals to bridge differences and foster understanding.', created_at=None, summarized=None), Edge(source=6, target=11, description=\"Abraham Lincoln's quote highlights the importance of understanding others to overcome dislike.\", created_at=None, summarized=None), Edge(source=7, target=12, description='Ben Casnocha introduced the idea that those we largely agree with are most likely to change our minds.', created_at=None, summarized=None), Edge(source=14, target=13, description=\"Clear's Law of Recurrence states that repeated ideas, even if false, gain belief proportionally to their repetition.\", created_at=None, summarized=None), Edge(source=8, target=13, description='Tyler Cowen advised focusing on good ideas rather than disputing wrong ones.', created_at=None, summarized=None)])" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + "Layer processed is: Topical Context\n", + "Layer graph is: nodes=[Node(id=1, description='Lee Parkin, 50, owner of the late terrier-spaniel cross Izzy, suffers from PTSD', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=2, description='Izzy, a terrier-spaniel cross owned by Lee Parkin, killed by an XL bully', category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=3, description='An XL bully attacked and killed Izzy during a walk with Lee Parkin in Doncaster', category='event', memory_type='episodic', created_at=None, summarized=None), Node(id=4, description='Doncaster, the location where Izzy was attacked and killed by an XL bully', category='location', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description=\"Naevia, Marie Hay's Siberian husky, survived an attack by two XL bullies with life-changing injuries\", category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description='Marie Hay, owner of Naevia, who survived an XL bully attack with mental and physical scars', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=7, description='Two XL bullies attacked Naevia on a beach in Redcar, causing severe injuries', category='event', memory_type='episodic', created_at=None, summarized=None), Node(id=8, description='Redcar Beach on the North Yorkshire coast, where Naevia was attacked by two XL bullies', category='location', memory_type='semantic', created_at=None, summarized=None), Node(id=9, description='XL bully, a breed involved in attacks on Izzy and Naevia, seen as dangerously out of control', category='animal', memory_type='semantic', created_at=None, summarized=None), Node(id=10, description='An incoming ban on XL bullies supported by Lee Parkin and Marie Hay but with reservations on its effectiveness', category='event', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='owned', created_at=None, summarized=None), Edge(source=1, target=3, description='witnessed', created_at=None, summarized=None), Edge(source=3, target=4, description='occurred in', created_at=None, summarized=None), Edge(source=6, target=5, description='owned', created_at=None, summarized=None), Edge(source=7, target=8, description='occurred in', created_at=None, summarized=None), Edge(source=7, target=5, description='targeted', created_at=None, summarized=None), Edge(source=9, target=3, description='involved in', created_at=None, summarized=None), Edge(source=9, target=7, description='involved in', created_at=None, summarized=None), Edge(source=10, target=9, description='targets', created_at=None, summarized=None), Edge(source=1, target=10, description='supports with reservations', created_at=None, summarized=None), Edge(source=6, target=10, description='supports', created_at=None, summarized=None)]\n", + "Layer processed is: Geographical Context\n", + "Layer graph is: nodes=[Node(id=1, description='Lee Parkin - 50-year-old dog owner, suffers from post-traumatic stress disorder after his dog was killed.', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=2, description='Izzy - Terrier-spaniel cross, pet dog owned by Lee Parkin for nearly 10 years, killed in dog attack.', category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=3, description='Doncaster - The location near where Lee Parkin was walking his dog Izzy when she was attacked.', category='location', memory_type='semantic', created_at=None, summarized=None), Node(id=4, description='XL Bully - Breed of dog that attacked and killed Izzy, described as dangerously out of control.', category='animal', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='Marie Hay - Dog owner whose Siberian Husky, Naevia, survived an attack but suffered life-changing injuries.', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description='Naevia - Siberian Husky owned by Marie Hay, survived a savage attack by two XL Bullies.', category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=7, description='Redcar Beach on the North Yorkshire coast - The location where Naevia was attacked by two XL Bullies.', category='location', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=3, description='resides in', created_at=None, summarized=None), Edge(source=1, target=2, description='was the owner of', created_at=None, summarized=None), Edge(source=2, target=4, description='killed by', created_at=None, summarized=None), Edge(source=5, target=6, description='is the owner of', created_at=None, summarized=None), Edge(source=6, target=7, description='attacked at', created_at=None, summarized=None), Edge(source=6, target=4, description='attacked by', created_at=None, summarized=None)]\n", + "Layer processed is: Temporal Context\n", + "Layer graph is: nodes=[Node(id=1, description='Lee Parkin, a 50-year-old man and owner of the dog Izzy.', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=2, description='Izzy, a terrier-spaniel cross owned by Lee Parkin for nearly 10 years.', category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=3, description='Attack by an XL bully on Izzy resulting in her death.', category='event', memory_type='episodic', created_at=None, summarized=None), Node(id=4, description='Lee Parkin was diagnosed with post-traumatic stress disorder after the attack.', category='condition', memory_type='semantic', created_at=None, summarized=None), Node(id=5, description='Marie Hay, owner of seven-year-old Siberian husky Naevia, also victim of an XL bully attack.', category='person', memory_type='episodic', created_at=None, summarized=None), Node(id=6, description='Naevia, a seven-year-old Siberian husky owned by Marie Hay, survived an attack by two XL bullies with life-changing injuries.', category='animal', memory_type='episodic', created_at=None, summarized=None), Node(id=7, description='The incoming ban on XL bullies that both Lee Parkin and Marie Hay support.', category='event', memory_type='semantic', created_at=None, summarized=None), Node(id=8, description='Anxiety and fear faced by Lee Parkin and Marie Hay when walking dogs post-attack.', category='condition', memory_type='semantic', created_at=None, summarized=None)] edges=[Edge(source=1, target=2, description='owned', created_at=None, summarized=None), Edge(source=1, target=3, description='involved in', created_at=None, summarized=None), Edge(source=2, target=3, description='killed in', created_at=None, summarized=None), Edge(source=1, target=4, description='diagnosed with', created_at=None, summarized=None), Edge(source=5, target=6, description='owned', created_at=None, summarized=None), Edge(source=6, target=3, description='injured in', created_at=None, summarized=None), Edge(source=1, target=7, description='supports', created_at=None, summarized=None), Edge(source=5, target=7, description='supports', created_at=None, summarized=None), Edge(source=1, target=8, description='experiences', created_at=None, summarized=None), Edge(source=5, target=8, description='experiences', created_at=None, summarized=None)]\n" + ] } ], "source": [ - "layer_decomposition" + "layer_2_graph = create_layer_graph(input_article_two, cognitive_layers_two)" ] }, { "cell_type": "code", - "execution_count": null, - "id": "15dc7863-0f4c-47ae-89ef-2656e8478249", - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "code", - "execution_count": 19, + "execution_count": 175, "id": "58644c64-7ef0-415f-8e41-e2edcf5fd15b", "metadata": {}, "outputs": [], @@ -786,7 +864,17 @@ "import uuid\n", "from datetime import datetime\n", "\n", - "def create_user_content_graph(user_id, custom_user_properties=None, additional_categories=None, default_fields=None):\n", + "def create_user_content_graph(user_id, custom_user_properties=None, required_layers=None, default_fields=None, existing_graph=None):\n", + "\n", + " category_name = required_layers.dict()['name']\n", + " subgroup_names = [subgroup['name'] for subgroup in required_layers.dict()['cognitive_subgroups']]\n", + "\n", + " \n", + " # Construct the additional_categories structure\n", + " additional_categories = {\n", + " category_name: subgroup_names\n", + "}\n", + "\n", " # Define default fields for all nodes if not provided\n", " if default_fields is None:\n", " default_fields = {\n", @@ -809,73 +897,73 @@ " if additional_categories:\n", " content_categories.update(additional_categories)\n", "\n", - " # Create a new MultiDiGraph\n", - " G = nx.MultiDiGraph()\n", + " G = existing_graph if existing_graph else nx.MultiDiGraph()\n", "\n", - " # Add the user node with properties\n", - " G.add_node(user_id, **user_properties)\n", + " # Check if the user node already exists, if not, add the user node with properties\n", + " if not G.has_node(user_id):\n", + " G.add_node(user_id, **user_properties)\n", "\n", - " # Add content category nodes and their edges\n", + " # Add or update content category nodes and their edges\n", " for category, subclasses in content_categories.items():\n", " category_properties = {**default_fields, 'type': 'category'}\n", - " G.add_node(category, **category_properties)\n", - " G.add_edge(user_id, category, relationship='created')\n", "\n", - " # Add subclass nodes and their edges\n", + " # Add or update the category node\n", + " if not G.has_node(category):\n", + " G.add_node(category, **category_properties)\n", + " G.add_edge(user_id, category, relationship='created')\n", + "\n", + " # Add or update subclass nodes and their edges\n", " for subclass in subclasses:\n", - " unique_id = str(uuid.uuid4())\n", - " subclass_node_id = f\"{subclass} - {unique_id}\"\n", - " subclass_properties = {**default_fields, 'type': 'subclass', 'content': subclass}\n", - " G.add_node(subclass_node_id, **subclass_properties)\n", - " G.add_edge(category, subclass_node_id, relationship='includes')\n", + " # Using both category and subclass names to ensure uniqueness within categories\n", + " subclass_node_id = f\"{category}:{subclass}\"\n", + "\n", + " # Check if subclass node exists before adding, based on node content\n", + " if not any(subclass == data.get('content') for _, data in G.nodes(data=True)):\n", + " subclass_properties = {**default_fields, 'type': 'subclass', 'content': subclass}\n", + " G.add_node(subclass_node_id, **subclass_properties)\n", + " G.add_edge(category, subclass_node_id, relationship='includes')\n", "\n", " return G\n", + "\n", + " # # Add content category nodes and their edges\n", + " # for category, subclasses in content_categories.items():\n", + " # category_properties = {**default_fields, 'type': 'category'}\n", + " # G.add_node(category, **category_properties)\n", + " # G.add_edge(user_id, category, relationship='created')\n", + "\n", + " # # Add subclass nodes and their edges\n", + " # for subclass in subclasses:\n", + " # unique_id = str(uuid.uuid4())\n", + " # subclass_node_id = f\"{subclass} - {unique_id}\"\n", + " # subclass_properties = {**default_fields, 'type': 'subclass', 'content': subclass}\n", + " # G.add_node(subclass_node_id, **subclass_properties)\n", + " # G.add_edge(category, subclass_node_id, relationship='includes')\n", + "\n", + " # return G\n", "\n" ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 168, "id": "dd3f0e55-9f9d-4804-9ad6-31afd2088ab5", "metadata": {}, "outputs": [], "source": [ - "category_name = required_layers.dict()['name']\n", - "\n", - "# Extract the names of the cognitive subgroups\n", - "subgroup_names = [subgroup['name'] for subgroup in required_layers.dict()['cognitive_subgroups']]\n", - "\n", - "# Construct the additional_categories structure\n", - "additional_categories = {\n", - " category_name: subgroup_names\n", - "}\n", - "\n" + "G = None" ] }, { "cell_type": "code", - "execution_count": 44, + "execution_count": null, "id": "2cc7c3bb-7cc0-453b-beab-2983a703ccda", "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'Natural Language Text': ['Articles, essays, and reports']}" - ] - }, - "execution_count": 44, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "additional_categories" - ] + "outputs": [], + "source": [] }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 170, "id": "199ef3ab-5e73-40d2-b531-6a402edf3f17", "metadata": {}, "outputs": [ @@ -884,10 +972,10 @@ "output_type": "stream", "text": [ "Nodes in the graph:\n", - "[('user123', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'username': 'exampleUser', 'email': 'user@example.com'}), ('Temporal', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Historical events - fe2b09db-93cb-4763-b186-5e08aa83b2f0', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Historical events'}), ('Schedules and timelines - 9584f7c2-bb9e-4b8d-9bf5-78545e705648', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Schedules and timelines'}), ('Positional', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Geographical locations - a695710f-9a71-4949-81a4-773e7c693802', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Geographical locations'}), ('Spatial data - 8ee3d6fb-1919-4cdb-978a-fe27d1456cd5', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Spatial data'}), ('Propositions', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Hypotheses and theories - 9e8d48b9-0a75-47f5-a113-b5c28cf6484d', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Hypotheses and theories'}), ('Claims and arguments - 0d6d809c-5858-4998-a959-3cd9494f5e24', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Claims and arguments'}), ('Personalization', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('User preferences - f84f9b30-42dd-47aa-bac0-170442371a80', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'User preferences'}), ('User information - 067b9536-8ce2-4cd2-ae9b-ca0771e79580', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'User information'}), ('Natural Language Text', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'category'}), ('Articles, essays, and reports - 97841c38-3dc7-4964-8b0c-ba8875a3b4ea', {'created_at': '2024-02-25 17:25:52', 'updated_at': '2024-02-25 17:25:52', 'type': 'subclass', 'content': 'Articles, essays, and reports'})]\n", + "[('user123', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'username': 'exampleUser', 'email': 'user@example.com'}), ('Temporal', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'category'}), ('Temporal:Historical events', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Historical events'}), ('Temporal:Schedules and timelines', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Schedules and timelines'}), ('Positional', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'category'}), ('Positional:Geographical locations', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Geographical locations'}), ('Positional:Spatial data', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Spatial data'}), ('Propositions', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'category'}), ('Propositions:Hypotheses and theories', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Hypotheses and theories'}), ('Propositions:Claims and arguments', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'Claims and arguments'}), ('Personalization', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'category'}), ('Personalization:User preferences', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'User preferences'}), ('Personalization:User information', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'User information'}), ('Natural Language Text', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'category'}), ('Natural Language Text:News stories and blog posts', {'created_at': '2024-02-27 18:11:57', 'updated_at': '2024-02-27 18:11:57', 'type': 'subclass', 'content': 'News stories and blog posts'})]\n", "\n", "Edges in the graph:\n", - "[('user123', 'Temporal', {'relationship': 'created'}), ('user123', 'Positional', {'relationship': 'created'}), ('user123', 'Propositions', {'relationship': 'created'}), ('user123', 'Personalization', {'relationship': 'created'}), ('user123', 'Natural Language Text', {'relationship': 'created'}), ('Temporal', 'Historical events - fe2b09db-93cb-4763-b186-5e08aa83b2f0', {'relationship': 'includes'}), ('Temporal', 'Schedules and timelines - 9584f7c2-bb9e-4b8d-9bf5-78545e705648', {'relationship': 'includes'}), ('Positional', 'Geographical locations - a695710f-9a71-4949-81a4-773e7c693802', {'relationship': 'includes'}), ('Positional', 'Spatial data - 8ee3d6fb-1919-4cdb-978a-fe27d1456cd5', {'relationship': 'includes'}), ('Propositions', 'Hypotheses and theories - 9e8d48b9-0a75-47f5-a113-b5c28cf6484d', {'relationship': 'includes'}), ('Propositions', 'Claims and arguments - 0d6d809c-5858-4998-a959-3cd9494f5e24', {'relationship': 'includes'}), ('Personalization', 'User preferences - f84f9b30-42dd-47aa-bac0-170442371a80', {'relationship': 'includes'}), ('Personalization', 'User information - 067b9536-8ce2-4cd2-ae9b-ca0771e79580', {'relationship': 'includes'}), ('Natural Language Text', 'Articles, essays, and reports - 97841c38-3dc7-4964-8b0c-ba8875a3b4ea', {'relationship': 'includes'})]\n" + "[('user123', 'Temporal', {'relationship': 'created'}), ('user123', 'Positional', {'relationship': 'created'}), ('user123', 'Propositions', {'relationship': 'created'}), ('user123', 'Personalization', {'relationship': 'created'}), ('user123', 'Natural Language Text', {'relationship': 'created'}), ('Temporal', 'Temporal:Historical events', {'relationship': 'includes'}), ('Temporal', 'Temporal:Schedules and timelines', {'relationship': 'includes'}), ('Positional', 'Positional:Geographical locations', {'relationship': 'includes'}), ('Positional', 'Positional:Spatial data', {'relationship': 'includes'}), ('Propositions', 'Propositions:Hypotheses and theories', {'relationship': 'includes'}), ('Propositions', 'Propositions:Claims and arguments', {'relationship': 'includes'}), ('Personalization', 'Personalization:User preferences', {'relationship': 'includes'}), ('Personalization', 'Personalization:User information', {'relationship': 'includes'}), ('Natural Language Text', 'Natural Language Text:News stories and blog posts', {'relationship': 'includes'})]\n" ] } ], @@ -903,7 +991,7 @@ "# \"Natural Language Text\": [\"Articles, essays, and reports\", \"Books and manuscripts\"]\n", "# }\n", "\n", - "G = create_user_content_graph(user_id, custom_user_properties, additional_categories)\n", + "G = create_user_content_graph(user_id, custom_user_properties, required_layers_one)\n", "\n", "# Accessing the graph\n", "print(\"Nodes in the graph:\")\n", @@ -914,15 +1002,95 @@ }, { "cell_type": "code", - "execution_count": null, - "id": "4dab2ff0-0d12-4a00-a4e4-fb901e701bd3", + "execution_count": 171, + "id": "b3160a1d-a6ea-40ce-a521-37ad26d31ffb", "metadata": {}, - "outputs": [], - "source": [] + "outputs": [ + { + "data": { + "text/plain": [ + "CognitiveCategory(name='Natural Language Text', cognitive_subgroups=[CognitiveLayerSubgroup(id=1, name='News stories and blog posts', data_type='TEXT'), CognitiveLayerSubgroup(id=2, name='Personal narratives and stories', data_type='TEXT')])" + ] + }, + "execution_count": 171, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "required_layers_two" + ] }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 172, + "id": "b254dfc9-ce85-4175-9d1e-c0f1ede67e3b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "MultiDiGraph with 15 nodes and 14 edges\n" + ] + } + ], + "source": [ + "print(G)" + ] + }, + { + "cell_type": "code", + "execution_count": 167, + "id": "a77b4f24-3046-4ab6-9ba1-c802096498df", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Personal narratives and stories'" + ] + }, + "execution_count": 167, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "required_layers_two.dict()['cognitive_subgroups'][1]['nam" + ] + }, + { + "cell_type": "code", + "execution_count": 177, + "id": "4dab2ff0-0d12-4a00-a4e4-fb901e701bd3", + "metadata": {}, + "outputs": [], + "source": [ + "B = create_user_content_graph(user_id, custom_user_properties, required_layers_two, existing_graph=G)" + ] + }, + { + "cell_type": "code", + "execution_count": 178, + "id": "627d42fd-d2ce-4ccd-a2a1-2f7ac2f463cf", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "MultiDiGraph with 16 nodes and 15 edges\n" + ] + } + ], + "source": [ + "print(B)" + ] + }, + { + "cell_type": "code", + "execution_count": 179, "id": "512f15be-0114-4c8c-9754-e82f2fa16344", "metadata": {}, "outputs": [ @@ -930,7 +1098,7 @@ "data": { "text/html": [ "\n", - "