Trying to sync internal database with a third party database but need to only update ones that have changed instead of rewriting the entire collection

Hi, I have a question that's hard to explain. Right now, we have a third party catalog of products that I want to keep in sync with my own internal catalog of products (using MongoDB). For example, if the person who owns the third party catalog, updates product A, B, and C or adds a new product D, I need to be able to tell if A,B,C has changed and it's updated and then add product D. The only problem is I have these products indexed on Algolia as well, so I need to know the exact products that have been added or/and updated instead of just completely just doing a bulk upsert, because I can't keep reindexing my algolia index every single time because it cost so much, so I just need to only add and/or update the objects that have changed or have been added.

Submitted May 21, 2020 at 04:54PM by wastedtimez