Best architecture for repeatedly sourcing data

Hey node community!​Long time lurker here, but recently ran into a situation where I could use some advice. For a startup of mine, I'm laying out the architecture for the MVP.​Essentially, our app sources data from a public API that has regular updates to its routes with the status of the returned objects. These updates occur every ~5 minutes, depending on the specific resource we're fetching. I currently have a react server deployed on a DigitalOcean droplet, as well as our (currently monolithic, but not that large) API deployed on another droplet. The database we're choosing for the MVP is a DO managed postgres DB, which we're using knex and objection.js to connect to and interface with.​The goal is to have the react server never reach out to the public API, and the DO-hosted Node API release browser notifications/push notifications eventually whenever a specific model is updated. I plan on doing this using triggers in the fetches on the API side itself, but can't think of a good way to regularly trigger sourcing/diffing the data.​Long story short (too late) I'm wondering how you would handle the regular data sourcing. My current solution uses Lambda triggered at regular intervals, but I'm trying to get the whole stack off of AWS. Are there any solutions you'd recommend reading into for this?​Thanks for reading guys, keep building!

Submitted July 15, 2019 at 05:33AM by DarthPlagueisTheWhys