I have a website that gets its content from web scraping. The website and the crawler are both written in python.I'm rebuilding the website in node as a SPA with a REST api in order to improve the website and to learn about building web applications with node and mongodb.What is my best bet for being able to keep using the python web crawler but have it insert data directly into mongo?the main problem I'm having is not being able to define a common schema for python & node.I've considered the possibility of having the web scraper not add data directly to the database but communicate via the api although I feel like that would needlessly slow it down, making one request to see if an item exists, and another one to add it.what should I be doing here?
Submitted February 06, 2016 at 10:57PM by davegri
No comments:
Post a Comment