I have a module that I'd like to run multiple instances to query different urls and process data as fast as possible. The server/api has a limit of 10 connections so I can't exceed that but when I run this code it may need to query as many as 25 urls. What is the best way to handle this on a server with 2 cores?I've looked into child processes but I'm not sure if this is the best way forward. Should I spawn 10 child processes and the 2-core server will figure out how to juggle them? Should I only spawn 2 child processes that each can handle 5 connections at a time and then write code that feeds each process a new url to query when it completes one of its existing url queries?Or, are both of my ideas garbage and I'm insulting the wonderful world of node? :D I am but a lowly designer that enjoys messing with node in my free time. Thanks!
Submitted June 11, 2017 at 03:36AM by yourfavorite
No comments:
Post a Comment