Thursday 25 July 2019

What the heck is a job queue in node?

I recently was asked to create a job queue as a code example for a potential employer. The problem was simple, or so I thought. Create an api that can take in a single url. The url should then be entered into a job queue for processing. The api returns a job id and another endpoint can be used to see if the job has finished.I assumed this was straight forward, create a module that generates a queue. Create a method on the module export that allows you to add an item to said queue. Allow the queue to decide if the item should be processed or put in line with other jobs.However, I was very kindly told this:it didn't really have a job queue. The endpoint POST route is responsible for the execution that grabs the HTML. So it basically doesn't even satisfy the requirements of the challenge.but it doesn't. Like all my endpoint does is say Scraper.add(url); Scraper being the name I gave to the queue because it literally only processes jobs that scrape the web. This doesn't create the queue, it doesn't do any of the processing, it just adds something to the queue.Google has been no help here because it's only convinced me even more that I did setup a queue. Can someone with more understanding of what exactly this term means help me understand what the fudge I did wrong.Too big to post the whole thing here but if you'd like to see the code it's here: https://github.com/ktranel/WebScraper

Submitted July 26, 2019 at 02:44AM by blindly_running

No comments:

Post a Comment