simplest but effective way to create a spider in node?

I want a long running (daemon perhaps?) that will continuously crawl web sites collecting links and following them, etc.Not sure the best design to use but it needs to be simple enough for me to implement but effective enough to produce results.

Submitted February 12, 2017 at 06:39AM by agentf90