Explain why initializing your weights with random orthogonal matrices might help mitigate the problem of exploding gradients in your deep neural network. Be sure to discuss any tradeoffs or limitations of this approach.

Initializing with random orthogonal matrices can help mitigate exploding gradients by allowing each layer to be scaled appropriately for its specific nonlinearity. This is because orthogonal matrices preserve the norm of vectors, ensuring that gradients are not amplified during backpropagation. However, this approach requires careful selection of the scaling factor g to account for the specific nonlinearity at each layer.