Lightning AI interview question

How does a gpu work? How are operations carried out in pytorch?