How best to handle data storage and processing?

Hi all,So... I have a multi tiered application which simply consumes data streams, stores the data and then allows users to view information regarding that data.Currently I am storing the data in MongoDB, however when it comes to reporting back, even simple analytics, I'm not confident MongoDB is the best solution, or at least, how I'm querying it is probably not the best solution. Data being stored can vary from a few documents per minute, to tens of thousands of documents per minute, which needs to be accessible as closet to immediate as possible, so users get the latest information.On the UI for the application, we're writing a lot of code to produce what could be done very easily using a relational database such as MySQL or MSSQL, i.e.model.findOne(id).exec().then(function(item) { if (item) { model2.count({ itemId: id, someCondition: true }).exec().then(function(count) { item.count = count; return resolve(item); }); } return reject(); }); which could be:select model.*, (select count(*) from model2 where itemId = model.id where someCondition = true) as `count` from model where id = ? That's an extremely easy example, but it shows that I have to write a number of lines of code, for something that can be achieved with one simple query. It gets messier the more complex the queries get.I am currently looking into Sequelize and PostgreSQL, however not sure that's going to be as scalable as I want.Any thoughts or advice on how best to proceed from here?Cheers

Submitted January 05, 2016 at 11:08AM by gRoberts84