Friday, 14 August 2020

How can I convert this Python script to do the same but with Node? I'm somewhat lost.

Hi all! I've made a relatively simple web scraper that will use POST to log in, and BeautifulSoup to parse html data and extract what I need. The code so far is:from requests_html import HTMLSession login_url = 'https://...log-in.taf' sim_list_url = 'https://.../listview.taf' payload = {"_action": "login", "mobile": "email", "mypass": "pass"} def get_sim_info(): with HTMLSession() as session: login_page = session.get(login_url) login_page = session.post(login_url, data=payload) sim_list = session.get(sim_list_url) sim_table = sim_list.html.find( '#myTable > tbody:nth-child(2)', first=True) rows = sim_table.find('tr') sim_database = {} for row in rows: mobile_number = row.find('a')[0].full_text # absolute_links is a set so convert to list with comprehension to access value directly activation_page = [link for link in row.find( 'a')[0].absolute_links][0] sim_iccid = row.find('td')[1].full_text # Ordering of parameters matters for activation code, sim number first, then mobile def activation_code(sim_iccid, mobile_number): return ''.join([a for x, y in zip(sim_iccid[::-1], mobile_number[::-1]) for a in (x, y)])[:6] sim_database[sim_iccid] = {'mobile': mobile_number, 'activation_code': activation_code(sim_iccid, mobile_number), 'activation_page': activation_page} print(f"Grabbed {len(sim_database)} SIM cards from list table") return sim_database get_sim_info() But to make this work better at work, I should make it in Node, as that's what we use. But I'm utterly lost. I've heard that the request library is deprecated, so I've been staying away from it and have been trying to use axios, but I'm finding it super confusing.Could someone give me some pointers on how I'd go about handling the session and redirects and posting the login data? I think I could use JSoup to parse the html data later, but the issue right now is even getting to that stage. Here's what I have for Node so far:const axios = require('axios'); const login_url = "https://.../log-in.taf"; // redirects here after logging in const account_url = "https://.../myaccount.taf"; const sim_list_url = "https://.../listview.taf"; const payload = { _action: "login", mobile: "email", mypass: "pass", }; async function makeRequest() { axios.post(login_url, { payload }, { withCredentials: true }) .then((response) => { console.log(response.status) }) .catch((error) => { console.error(error); }) makeRequest() Some help would be very much appreciated!

Submitted August 14, 2020 at 04:16PM by theoriginal123123

No comments:

Post a Comment