Amazon interview question

How to build a a scalable web crawling platform