Help with Node Transform Streams

I have an API endpoint that is parsing RSS Feeds, stripping out relevant data (the Transform step), and then sending that data back to the client. As of right now I'm able to accomplish all of this but the response is not sent to the client until after the Transform step finishes which is resulting in long response times. I think this is a back pressure problem as during the Transform step I am converting Objects to JSON which is probably too slow.​const FeedParser = require('feedparser'); //RSS parsing Library const fetch = require('node-fetch'); const { Transform } = require('stream'); const jsonTransformer = new Transform({ writableObjectMode: true, transform(chunk, encoding, callback) { try { // Not sure yet if this is the proper way to handle 'end of data' if(chunk === null){ this.push(null); } let {title, enclosures, author, date, link, origlink} = chunk let episode = chunk['itunes:episode'] let data = {title, author, date, enclosures, link, origlink, episode} let podcastData = JSON.stringify(data); this.push(podcastData) callback(); } catch (error) { callback(error); this.push(null) } } }) export default async function handler(req, res) { //Feedparser is a Transform stream that operates in Object Mode const feedparser = new FeedParser(); let {feedUrl} = req.body; const podcastRequest = await fetch(feedUrl); podcastRequest.body.pipe(feedparser).pipe(jsonTransformer).pipe(res); } I'm using the .pipe() method to send data to the response stream so I assumed it would just start sending off data as soon as it's ready. But this doesn't seem to be the case. Does anyone have an idea of why?

Submitted September 03, 2020 at 02:06AM by ForeverUnfortunate