We would choose SVD over PCA because SVD can decompose all matrices, regardless of their properties. This is particularly useful when dealing with numerical instability or matrices that cannot be decomposed into eigenvalues and eigenvectors. However, this comes at the cost of increased computational complexity and memory usage.