I’m trying to run a large-scale node.js puppeteer application. The code for a single thread works fine, but I’ve run into a bottleneck. My computer can only handle about 20 simultaneous threads reliably. Ideally, I would be able to run 200+ threads.I’ve been told I can accomplish this with Amazon SQS, but I really have no idea what it is. The SQS docs aren’t that clear to me either.A potential “solution” i was thinking about is programmatically creating a lot of separate heroku instances, pushing my puppeteer code to all of them, and having them all run on the cloud. Does heroku even have this capability? I’ve only ever seen heroku used in websites, and code only actually runs when the page is opened.How would you go about accomplishing this?
Submitted March 02, 2020 at 06:03PM by rpg7toter
No comments:
Post a Comment