gmakstutis/cognee
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Adapt chunk_by_sentence to isomorphic chunk_by_word

Browse source
This commit is contained in:
Leon Luithlen 2024-11-13 11:52:20 +01:00
parent c054e897a3
commit ab55a73d18
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
cognee/tasks/chunks/chunk_by_sentence.py
View file

@ -11,7 +11,7 @@ def chunk_by_sentence(data: str):
word_count = 0
for (word, word_type) in chunk_by_word(data):
sentence += (" " if len(sentence) > 0 else "") + word
sentence += word
word_count += 1
if word_type == "paragraph_end" or word_type == "sentence_end":