Saturday 26 September 2020

[HELP] Count URLs and track execution time in node.js CLI app

Hi! I have a node.js CLI program that takes a file, looks for URLs and then makes a head request to see the response status. The input is a file with URLs. One line can have more than 1 URL.The main functionality works but I want to be able to keep track of how long it takes to complete everything (starting from readStream data event to end) as well as count total URLs assessed (starting from readStream data event to end). I don't know how to go about keeping track of time or total URLs. I tried using a normal counter and console.time but they just print at the beginning. (I think because of Node.js' non-blocking loop? minimal understanding of how that works).How can I achieve that without changing the whole code? I would appreciate your help.My code: (this is runnable. Just npm install axios. You can use a sample.txt file with 1-2 URLs as input and change argv.file to your filepath).// Create read stream object to read the file in chunks. const readStream = fs.createReadStream(argv.file); let urlList; readStream.on("data", (chunk) => { // Get all the URL links from the file. urlList = chunk .toString() .match(/(http|https)(:\/\/)([\w+\-&@`~#$%^*.=/?:]+)/gi); }); readStream.on("end", () => { // Check each link's status asynchronously urlList.forEach(async (url) => { try { const urlResponse = await axios.head(url); if (urlResponse.status == 200) { console.log( chalk.green.bold(`[SUCCESS] Status: [${urlResponse.status}] ${url}`) ); } else if (urlResponse.status == 400 || urlResponse.status == 404) { console.log( chalk.red.bold(`[FAIL] Status: [${urlResponse.status}] ${url}`) ); } else { console.log( chalk.grey.bold(`[UNKNOWN] Status: [${urlResponse.status}] ${url}`) ); } } catch (error) { console.error(chalk.grey.bold(`[UNKNOWN] Status: [UNKNOWN] ${url}`)); } }); }); readStream.on("error", (e) => { console.error(e); });

Submitted September 26, 2020 at 10:52PM by fothingers

No comments:

Post a Comment