Design an architecture that leverages a pre-trained network as a teacher for your student network, and explain how this approach can help improve the robustness of your model to noisy data.

Use a wide, shallow network (e.g. 10 layers with many units per layer) as the teacher, and train it on a subset of the dataset until convergence. Then use the weights of the teacher as initialization for the student network, which has fewer units per layer but more layers. Jointly train the student network to classify images from the full dataset.