Friday 10 February 2017

Nightmare JS updating MongoDB + Mongoose with retrieved data not working.

I am looking to scrape data from specific URLs and updating an existing data in MongoDB. I am also using CRON (not sure if that has any affect on the code).Below is the code:module.exports = new CronJob('0 */59 * * * *', function() { nightmare .goto('http://ift.tt/2lx8AWn') .wait(2000) .evaluate(function() { var itemS = []; var itemList = document.querySelectorAll('.item-list'); for(var i = 0; i < item-list.length; i++) { var item = {}; item['name'] = itemList[i].querySelector('.item-title').innerHTML; item['image'] = itemList[i].querySelector('.item-image').src; itemS.push(item); } return itemS; }) //.end() -- removed because CRON does not restart .then(function(result) { var id = mongoose.Types.ObjectId("589b8d8fe860591071d4e2c6"); mongooseModel.findByIdAndUpdate(id, {$set: {key: result}}, function(err, res) { if(err) { console.log(err); } else { console.log('success!'); } }); console.log(result); }) .catch(function(error) { console.error('Search failed:', error); }); }, null, true); The results are console.log(ged) (console.log(result)) but the console.log("success!") callback does not execute.Also, unrelated, when I pull images from some sites, I'm getting Data URLs instead of the img url. I tried putting the data URL directly into a tag, but the image shows up blank -- I also tried decoding the image, using online resources, but the image still does not render. any possible solutions for this?Stackoverflow

Submitted February 10, 2017 at 05:51PM by thederelict87

No comments:

Post a Comment