cognee/cognee/modules/pipelines/operations/run_parallel.py
Boris 14555a25d0
feat: pipelines and tasks (#119)
* feat: simple graph pipeline

* feat: implement incremental graph generation

* fix: various bug fixes

* fix: upgrade weaviate-client

---------

Co-authored-by: Vasilije <8619304+Vasilije1990@users.noreply.github.com>
2024-07-20 16:49:00 +02:00

12 lines
468 B
Python

from typing import Any, Callable, Generator
import asyncio
from ..tasks.Task import Task
def run_tasks_parallel(tasks: [Task]) -> Callable[[Any], Generator[Any, Any, Any]]:
async def parallel_run(*args, **kwargs):
parallel_tasks = [asyncio.create_task(task.run(*args, **kwargs)) for task in tasks]
results = await asyncio.gather(*parallel_tasks)
return results[len(results) - 1] if len(results) > 1 else []
return Task(parallel_run)