You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I test the following two methods, method_1 can show multiprocessing is involved for task dispatching. While in method_2, all tasks are involved in a single process, can anyone show me a way out for involving multiprocesses in method_2? Thx in advanced.
Details
async def get(url):
async with request("GET", url) as response:
result = await response.text("utf-8")
logger.info(len(result))
return result
async def method_1():
urls = ["https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev"]
async with amp.Pool() as pool:
async for result in pool.map(get, urls):
logger.info(len(result))
async def method_2():
pool_tasks = []
calls_list = ["https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev", "https://jreese.sh", "https://noswap.com", "https://omnilib.dev"]
async with amp.Pool() as pool:
for call in calls_list:
pool_tasks.append(pool.apply(get, args=[call]))
[await _ for _ in tqdm(asyncio.as_completed(pool_tasks), total=len(pool_tasks), ncols=90, desc="total", position=0, leave=True)]
OS: Mac
Python version: 3.8.2
aiomultiprocess version: 0.9.0
Can you repro on master?
Can you repro in a clean virtualenv?
The text was updated successfully, but these errors were encountered:
Description
I test the following two methods, method_1 can show multiprocessing is involved for task dispatching. While in method_2, all tasks are involved in a single process, can anyone show me a way out for involving multiprocesses in method_2? Thx in advanced.
Details
The text was updated successfully, but these errors were encountered: