Thursday 30 April 2020

[help] i'm trying to store into mongoDB infos from reddit 'API' but i can't get past first results!

hello everybody!i'm trying to learn some nodejs, and i was trying to store into a mongoDB (a free atlas cluster), data from reddit.the data that i'm getting is from https://www.reddit.com/r/pics/top.json?t=weekto go to the next page i can add to the end of the link &after= plus the info that i get from the json itself (it's an alphanumeric string, like t3_gaea5j. i want to cycle and get as many results as i want, but after the first result the data that i pull out is always the same.this is the promise that i use to get the json const redditjson = new Promise((resolve,reject)=>{ i++; json = ''; let parsed =''; console.log(); //var options = new URL('https://www.reddit.com/r/pics/top.json?t=weer next=' r next); https.get(url, (resp) => { resp.on('data', (chunk) => { json += chunk; resp.on('end', () => { parsed = JSON.parse(json); next = parsed.data.after; return resolve (parsed); }); }); }); and this is the function that i use to fetch what i needasync function connect() { app.listen(3000, () => { MongoClient.connect(CONNECTION_URL, { useNewUrlParser: true }, (error, client) => { if (error) { throw error; } database = client.db(DATABASE_NAME); collection = database.collection("pic"); console.log("Connected to " + DATABASE_NAME + "`!"); // upload(redditjson); for (let index = 0; index < 3; index++) { redditjson.then(result=> { console.log(index); console.log(next); next = result.data.after; url = ('https://www.reddit.com/r/pics/top.json?t=week&after=' + next); result.data.children.forEach(element => { redditObj = { 'upvote': element.data.ups, 'subreddit': element.data.subreddit, 'utcPost': element.data.created_utc, 'link': element.data.permalink, 'title': element.data.title, 'type': element.data.post_hint }; console.log(redditObj); arrObj.push(redditObj); }); //upload(arrObj, collection); redditObj = []; arrObj={}; }); } }); }); } code is a little bit messy because i tried a lot of things before coming to ask here, so sorry if it doesn't make any sense to you!things that i tried so far :- eventHandler-promises-calling what's inside the promise as a function inside connect();-tried without async functionsthe first time loop it works, at the second one it just breaks, and gives me back nothing.thanks to everyone who wants to help someone who wants to learn nodejs!

Submitted April 30, 2020 at 08:02PM by christmas_fighters

No comments:

Post a Comment