[help needed] Custom body-parser?

Hi folks!I didn't know how to title this post because it is very specific to my case, I guess.I am sorry for the length of the post, but I don't see any solution, so I would like a hand.I have a Node.js (specifically Express.js) application that receives data from different sensors that record the number of people that pass in front of the sensors themselves.These sensors are of different kinds and send data in different formats, but in the end they all contain the same information. Some sensors send data in JSON (Content-Type: application/json) and some in plain text (Content-Type: text/plain).All of them send the same information, but due to the different formats the information needs to be accessed in a different way for each sensor.For instance, the sensor that sends in JSON sends an HTTP POST request whose body would be something like this:{ [some other info releated to the sensor that sent this data], "content" : [ { "from" : "2018-08-06T12:10:00Z", "to" : "2018-08-06T12:15:00Z", "passed": 10 }, { "from" : "2018-08-06T12:15:00Z", "to" : "2018-08-06T12:20:00Z", "passed": 9 }, { "from" : "2018-08-06T12:20:00Z", "to" : "2018-08-06T12:25:00Z", "passed": 14 }, ... ] } The sensors that sends in plain text, instead, would send an HTTP POST request whose body would be something like this:H201808061215P00010 H201808061220P00009 H201808061225P00014 where the numbers after the H represent the datetime and after the P represent the number of passages in front of the sensor.Now, both formats represent the same data and are fine, but I have a huge problem of performance with the plain text, especially if the body is big (which could happen if the sensor stays offline for sometime, keeps collecting the data and then sends the data all at once, yes it is not smart).Express.js provides the package body-parser which is useful to parse the body of an HTTP request. Also, the package is very performant even in case of huge HTTP payloads. In the case of the JSON request, bodyParser.json([options]) is extremely useful as parsing the body gives me a ready-to-use JSON object with the information from the sensor. But in the case of a plain/text HTTP payload, the bodyParser.text([options]) returns me a req.body that is just a string that I cannot easily access to extract the information, but needs further processing.Basically, in the case of a `text/plain payload, I find myself running bad synchronous code that does the following:... let arr=[]; data = req.body.trim().split(/\s+/); foreach(el in data){ let obj = { "datetime": moment.utc(el.substring(1, 13), 'YYYYMMDDHHmm'), "passed": parseInt(el.substring(14)) }; arr.push(obj); } // Now I have all the data available ... This code performs (seconds, which is unacceptable) extremely bad for big requests (e.g. bodies with thousands of lines ~30k), whereas the body parses remains super fast.What are the steps I could take to improve this bottleneck?

Submitted August 06, 2018 at 04:33PM by honestserpent