Just a bunch of iPython notebooks for future references.
Matrix Decomposition: Eigenvalue decomposition vs SVD
ICP: SVD vs Least Squares
We start with a typical least squares formulation and go through most useful methods for solving this type of minimization problems, such as Steepest Gradient Descent and the Gauss-Newton method. We derive everything from the way the problem looks like and use its Taylor expansion.