You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #4 has the idea of a bufferAhead helper that can create concurrency. It has some discussion about ways it might want to limit concurrency, & in that PR I've advocated for keeping bufferAhead simpler / complecting less concerns into it. But the need seems real to create a way to limit concurrency (the number of unresolved promises at any given time).
letpages=asyncIteratorOfUrls.map(u=>fetch(u)).limitConcurrency(2);// bad consumer we want to guard against.// more likely examples might be some kind of a thread pool.for(leti=0;i<10;++i){pages.next().then(()=>console.log("got page"));}
There's a number of jobs where I've created or been around folks making Staged Event Driven Architecture (SEDA) like structures, where there's a bunch of processors & queues in between them. Along with bufferAhead, the limitConcurrency(n) helper proposed here should allow fulfillment of those architectures quite quickly:
const[staticReq,dynamicReq]=packet.map(parse).limitConcurrency(3).bufferAhead(3).map(urlDispatch).limitConcurrency(3).bufferAhead(3)// imagining a fork/join helper to split/merge iterators.fork(staticOrDynamic)constgenerated=dynamic.map(generateDynamic).limitConcurrency(3).bufferAhead(3)const[hit,miss]=staticReq.map(checkCache).limitConcurrency(3).bufferAhead(3).fork(isCacheHit)constfilled=miss.map(loadFile).limitConcurrency(3).bufferAhead(3)constsent=join(generated,hit,filled).map(send).limitConcurrency(5).bufferAhead(5)
It's possible we could try to combine all these behaviors into one big helper, and the code above definitely looks a bit simple. But de-complecting the concerns feels like it creates more possibilities than trying to tackle everything together.
The text was updated successfully, but these errors were encountered:
In #4 has the idea of a
bufferAhead
helper that can create concurrency. It has some discussion about ways it might want to limit concurrency, & in that PR I've advocated for keepingbufferAhead
simpler / complecting less concerns into it. But the need seems real to create a way to limit concurrency (the number of unresolved promises at any given time).There's a number of jobs where I've created or been around folks making Staged Event Driven Architecture (SEDA) like structures, where there's a bunch of processors & queues in between them. Along with
bufferAhead
, thelimitConcurrency(n)
helper proposed here should allow fulfillment of those architectures quite quickly:It's possible we could try to combine all these behaviors into one big helper, and the code above definitely looks a bit simple. But de-complecting the concerns feels like it creates more possibilities than trying to tackle everything together.
The text was updated successfully, but these errors were encountered: