Limiting Concurrent Requests
Running all promises in parallel can overwhelm a server. Process items in batches or use a concurrency limiter.
Running all promises in parallel can overwhelm a server. Process items in batches or use a concurrency limiter.
async function batchProcess(items, batchSize, fn) {
for (let i = 0; i < items.length; i += batchSize) {
const batch = items.slice(i, i + batchSize);
await Promise.all(batch.map(fn));
}
}
await batchProcess(urls, 5, url => fetch(url));
// Or use a library: p-limit, bottleneck
Batch processing prevents rate-limiting and memory spikes when dealing with large datasets.
More in JavaScript