I have 3 tasks that are all time consuming. 1 is a web crawler, the second is analyzer that tries to match an image against other images, and the 3rd is a color analyzer of an image. Each is going to block for enough time that I don't want them all running on the same process.If I understand clustering correctly it won't really solve my problem because I want each task to have equal availability and if the crawler or the analyzer eats up all the room on the cluster than the color analyzer just waits.What I was thinking about doing was creating a child process for the 3 tasks so they can each run separately and just pass info back and forth. However, the same could be accomplished by just creating 3 different node instances and communicating via an api.In cases like this what would be seen as best practice or is there a 3rd option I'm not thinking of?
Submitted August 21, 2019 at 06:13PM by blindly_running
No comments:
Post a Comment