Strategies for Propagating Document Updates

I was once on a team fully on the MEAN stack. Our job was to write full-stack code from the database, server, and front-end and crush lots of story points! Things went well until our models became very complex, and the Node <-> MongoDB relationship started to get hairy. One particular problem was propagating updates. When I talk about propagating updates, I mean I have document/record A that document/record B depends on. When I make an update to A, B also needs to update. Keep in mind, the API was technically the arbiter to the database, so any and all updates on the model were through the server. Now, with that, we could have easily just, on a request to update document A, also fetch document B and make the update. Some questions get raise when you need to send a response to the client, but in any case it's a simple thing to not only implement, but test. Everything is serial and after stubbing out functions, very deterministic.Instead of doing the simple thing with propagating updates and just promise chaining, someone decided before my time to implement an event bus. When a document is updated, it sends an event to the event bus, and all Models essentially have listeners that will correctly make the updates. The problem ended up being testing. Instead of using a fake event bus that just deterministically stores your listeners and you call them when you need it, we used... the real event bus.Enter nightmares.Tests randomly failed due to the tests not completing in time, because of course the stack must be cleared before popping off a callback in the queue. Each time, our only answer was, "Let's increase the timeout!" THINK ABOUT THIS. The event bus adds callbacks to the queue through libuv, and there's no guarantee when libuv will push to the stack, nor is there an order to it. A failing test will say, "I failed after 5 seconds," yet after increasing the timeout to 10 seconds, if a test runs and other async tests are added to the queue before pushing it back even further, it means you have to recommit pushing it back, again. This was probably the most annoying part of our jobs considering you would work on a feature, then Travis wouldn't ok the build because a test failed in a code chunk that wasn't even touched.Now in this case, what would you do? Sadly, I had no power to convince the devs to refactor because no one ever wants to add technical debt in a startup where features matter. In retrospect, this was over-engineering. EventEmitters have their place for multiple events. They are the dual to iterators. Here we have a single event (a user making an update) and when it's done, we want to call a single callback (updating the second document). I could see why we may want a stream of that, but it's arguably worse from a practical standpoint since we now have to test it using lots of timeouts. I think the only bonus to eventEmitters is that you can vertically scale since the events, and their updates are atomic, but even that would have required some refactoring of the event bus and at the time, it was actually all running single-threaded.So I ask, how do you handle propagating updates? What are your thoughts on avoiding EventEmitters until absolutely necessary? What are your thoughts on using EventEmitters in a multi-threaded environment?

Submitted June 06, 2017 at 06:02PM by umib0zu