Monday, 18 May 2020

Consumer/producer model in nodejs

Hi all,Currently I am restructuring my project. Basically I send all my jobs to my consumer by producer, and process the jobs in consumer. (I send from parent process, and I forked two new child process as workers)But the catch is that all my jobs in consumer is just a request to third party api, so no cpu intensive works. And I used to use await for each request, and used a timeout to kill the worker once the request takes too long.But just after finishing that logic, I found myself absurdly wrong. Why I need to await all the things one by one in my consumer where I could just let the non-blocking queue handles more. I realized that I was just killing the advantage of using nodejs (ASYNC).So I decided to remove the await. But I am still feeling somewhat weird. What is the best practice for nodejs consumer that handles api calls?​Should I just use one worker to do this async works? And can I assume the worker will never hangs, so I don't need to actually kill the worker. And I actually don't need a process controller where to do health check like that?​Really appreciated for your thoughts on this!

Submitted May 18, 2020 at 04:54PM by GreatFireWallSucks

No comments:

Post a Comment