Criteo interview question

Design d'un web crawler