RSS, MSE, RMSE, RSE, TSS, R² and Adjusted R²

This post is written by courtesy of:

• Panna Pan
• R - Confused on Residual Terminology

以下假设 sample 有 $m$ 个 examples。

The Residual Sum of Squares (RSS) is the sum of the squared residualsPermalink

以下三个概念等价 (我无话可说)：

• RSS: Residual Sum of Squares
• SSR: Sum of Squared Residuals
• SSE: Sum of Squared Errors

The Mean Squared Error (MSE) is the mean of RSSPermalink

The Root Mean Squared Error (RMSE) is the square root of MSEPermalink

The Residual Standard Error (RSE) is the square root of $\frac{RSS}{\text{degrees of freedom}}$Permalink

where

• $p$ is the number of predictors
  • i.e. $p+1$ is the number of right-hand-side variables, including the intercept, in a regression model
• $m-p-1$ denotes the degrees of freedom.

The Total Sum of Squares (TSS) is related with variance and not a metric on regression modelsPermalink

where $\overline{y}$ is the sample mean.

Further we have $Var=\frac{TSS}{m-1}$

$R^2$ and Adjusted $R^2$Permalink

Chain ReactionPermalink

当趋向 overfitting 时（比如 predictor 增多，模型变 flexible 时）：

• RSS ↓
  • MSE ↓
  • RMSE ↓
  • 如果是 predicator 增多，那么 RSE 无法断定是上升还是下降
• TSS →
• $R^2$ ↑
• Adjusted $R^2$ 不好说（这正是 adjustment 的体现）