Tuesday 25 August 2020

[Node JS / Heroku] Can I use a Background worker to execute CPU intensive tasks and return the data to client? Or, should I use a child process?

I apologize if I mixed up child and background process.I'm fairly new to using background workers with Bull on Node JS, and was hoping to get some help on how to handle this. I'm currently using standard-1x dyno for web and worker process on heroku.I have a process in my server that retrieves the data from a DB and performs calculations on this data (CPU Intensive). The worker process is currently being used for handling background tasks such as database backup, send emails, etc. I'm able to add the CPU intensive task to the Bull queue and execute it on worker thread, but I'm having trouble with returning the data either to the main thread or to the client.I've tried sending the data over sockets from the worker thread, but that fails. Another approach would be to store it in Redis, and poll an endpoint for this result.Is there a better way to do this? Or should I drop moving the exec. to worker threads and use child process instead? If I'm using child process, what does that mean for my dyno? How should I change it's configuration?

Submitted August 25, 2020 at 03:29PM by the-ML-noob

No comments:

Post a Comment