Thursday 17 December 2015

How best to handle communication between my distributed systems?

Hey all,I've been banging my head against a brick wall for the past couple of days trying to find the best solution for this, however I'm not getting anywhere fast.So, I have a number of clustered processes that currently communicate via an AMPQ (RabbitMQ) messaging queue, which essentially queues jobs and data to be processed.Simplified overviewQueueUnfortunately I misunderstood how PUSH/WORKER queues work, so currently I send a PUSH message to initiate a data stream and to ensure that the message is requeued should anything go wrong, I don't acknowledge that PUSH message.The issue with this, is that any other PUSH messages, i.e. for another data stream or more importantly, the PUSH messages for the data it self, won't be delivered until I acknowledge all unacknowledged PUSH messages.Has anyone got any idea's on the best way to handle this? I'm looking at REQ/REP, however I'm not sure whether this will be any different.Thanks.

Submitted December 18, 2015 at 07:08AM by gRoberts84

No comments:

Post a Comment