Least Squares Linear Algebra Interview Guide
Least squares linear algebra interview guide for projections, residuals, design matrices, normal equations, QR/SVD intuition, and pitfalls.
Candidates connecting regression formulas to geometry and numerical methods.
Least squares is projection
In linear regression, fitted values are the projection of the outcome onto the column space of the design matrix under standard assumptions.
Residuals are orthogonal to fitted directions
At the least-squares solution, residuals are orthogonal to the included regressors. This gives geometric intuition for the normal equations.
Concrete example
Fitting returns on factors means finding coefficients that minimize squared residuals between observed returns and factor-implied returns.
Numerical method matters
Normal equations are easy to write but can be unstable under collinearity. QR or SVD-based methods may be preferable in numerical code.
Common mistakes
Candidates often jump from formula to coefficient interpretation. Strong answers mention design matrix, residuals, assumptions, and conditioning.
Practice the pattern
Use the LeetQuidity curriculum and calibration to turn this topic into a focused practice plan.