Linear Regression Is Actually a Projection Problem (Part 2: From Projections to Predictions)

Linear regression can be understood as a vector projection problem in a "column space" rather than simply fitting a line in a "feature space". In this alternative perspective, entire data columns for features and the target variable are treated as vectors, with each data point representing a dimension. The objective of regression then becomes finding a scalar multiple of the feature vector that creates a projection as close as possible to the target vector. This geometric approach simplifies the underlying mathematics, especially for multiple regression, compared to the traditional calculus-based method of minimizing squared errors. The model's intercept is introduced as a constant "base vector" to account for a baseline value when feature values are zero.