Monday 22 February 2016

Crons sabotaging my event loop

I've got a node script that needs to "check in/heartbeat" every 5sec. I'm doing this with a simple setInterval.This same node script handles cronjobs. I'm using node-cron and up to 5000 crons may fire at exactly the same time. The function that gets called when the cron fires is semi-CPU intensive (for a few hundred MS), so when these crons fire, my setInterval heartbeats don't check-in.In my tests, my setIntervals go silent for about 60sec.So, I figured that I'd write a test app that delegated the CPU intensive function to a cluster worker. So when the cron fires, the function now simply says worker.send('start')When the cluster worker is done with the CPU intensive task, it sends a message back to the master process saying 'done'.Now... My setIntervals is STILL go silent for about 40 seconds - even if I spawn off a worker for every CPU core (8) and send the CPU intense job to a random worker.Any thoughts on making this more efficient (assuming I can't randomize the cron firing timing a bit)?

Submitted February 22, 2016 at 04:34PM by TheNodist

No comments:

Post a Comment