Proofs of some formulas in Anderson's Statistics for Business and Economics

• (9.7) Sample Size for a One-Tailed Hypothesis Test about a Population Mean
• (10.7) Degree of Freedom: t Distribution with Two Independent Random Samples
• (13.12) Test Statistic for the Equality of k Population Means
• (13.14) SSE+SSTR=SST
• (13.16) Fisher's LSD
• (14.11) Relationship among SST, SSR, and SSE
• (14.13) Sample Correlation Coefficient
• (14.15) Mean Square Error (Estimate of $\sigma^2$)
• (14.19) Test Statistic for t Test for Significance in Simple Linear Regression
• (14.21) Test Statistic for F Test for Significance in Simple Linear Regression
• (14.27) Prediction Interval for y*
• (14.30) Standard Deviation of the ith Residual
• (14.33) Leverage of Observation of i
• (15.9) Adjusted Multiple Coefficient of Determination
• (18.8) Spearman Rank-Correlation Coefficient
• References

(9.7) Sample Size for a One-Tailed Hypothesis Test about a Population Mean

$$n=\frac{(z_{\alpha}+z_{\beta})^2 \sigma^2}{(\mu_0-\mu_a)^2}$$ In a two-tailed hypothesis test, use (9.7) with $z_{\alpha/2}$ replacing $z_{\alpha}$.

Proof: Note that we don't have to change $\beta$ in a two-tailed hypothesis.

(10.7) Degree of Freedom: t Distribution with Two Independent Random Samples

$$\text{d.f.}=\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{1}{n_1-1}\left(\frac{s_1^2}{n_1}\right)^2+\frac{1}{n_2-1}\left(\frac{s_2^2}{n_2}\right)^2}$$

Proof: See [Casella, p.409, exe.8.42].

(13.12) Test Statistic for the Equality of k Population Means

$$F=\frac{\text{MSTR}}{\text{MSE}}$$

Proof: This proof is from [Hogg, PSI, sec.9.3]. Recall that

$$\begin{array}{cccccc} \text{treatment 1} & \text{treatment 2} & \cdots & \text{treatment k} \\ x_{11} & x_{12} & \cdots & x_{1k} \\ x_{21} & x_{22} & \cdots & x_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n_1 1} & x_{n_2 2} & \cdots & x_{n_k k} \\ \downarrow & \downarrow & \cdots & \downarrow & \searrow \\ \overline{x}_1 & \overline{x}_2 & \cdots & \overline{x}_k & & \bar{\bar{x}} \end{array}$$ Recall the following formulas $$\begin{array}{rcl} \text{SSE} &\stackrel{\text{(13.11)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2=\sum_{j=1}^{k} (n_j-1)s_j^2 \\ \text{SSTR} &\stackrel{\text{(13.8)}}{=}& \sum_{j=1}^{k} n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ \text{SST} &\stackrel{\text{(13.13)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \\ \text{MSE} &\stackrel{\text{(13.10)}}{=}& \frac{\text{SSE}}{n-k} \\ \text{MSTR} &\stackrel{\text{(13.7)}}{=}& \frac{\text{SSTR}}{k-1} \\ \end{array}$$ At first, $$\begin{array}{ll} & \frac{\text{SST}}{n-1}=\frac{\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2}{n-1}\stackrel{\text{用全部來算sample variance}}{=}S^2 \\ \stackrel{\text{[Casella, p.218, thm.5.3.1.(c)]}}{\Rightarrow} &\frac{\text{SST}}{\sigma^2}=\frac{(n-1)S^2}{\sigma^2}\sim \chi^2_{n-1} \end{array}$$ Then $$\begin{array}{cl} & \frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{n_j-1}\stackrel{\text{用單行來算sample variance}}{=}S^2 \\ \stackrel{\text{[Casella, p.218, thm.5.3.1.(c)]}}{\Rightarrow}& \frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{\sigma^2}=\frac{(n_j-1)S^2}{\sigma^2}\sim \chi^2_{n_j-1} \\ \stackrel{\text{[Casella, p.219, lem.5.3.2.(b)]}}{\Rightarrow}& \frac{\text{SSE}}{\sigma^2}=\sum_{j=1}^{k}\frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{\sigma^2}\sim \chi^2_{(n_1-1)+(n_2-1)+\cdots +(n_k-1)}=\chi^2_{n-k} \\ \stackrel{\text{[Casella, p.155, thm.4.2.12]}}{\Rightarrow}& \frac{\text{SSTR}}{\sigma^2}=\frac{\text{SST}}{\sigma^2}-\frac{\text{SSE}}{\sigma^2}\sim \chi^2_{k-1} \\ \Rightarrow & \frac{\text{MSTR}}{\text{MSE}}=\frac{\text{SSTR}/(k-1)}{\text{SSE}/(n-k)}=\frac{(\text{SSTR}/\sigma^2)/(k-1)}{(\text{SSE}/\sigma^2)/(n-k)}\sim F(k-1, n-k) \end{array}$$

(13.14) SST=SSTR+SSE

$$\text{SST}=\text{SSTR}+\text{SSE}$$

Proof: Recall that

$$\begin{array}{cccccc} \text{treatment 1} & \text{treatment 2} & \cdots & \text{treatment k} \\ x_{11} & x_{12} & \cdots & x_{1k} \\ x_{21} & x_{22} & \cdots & x_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n_1 1} & x_{n_2 2} & \cdots & x_{n_k k} \\ \downarrow & \downarrow & \cdots & \downarrow & \searrow \\ \overline{x}_1 & \overline{x}_2 & \cdots & \overline{x}_k & & \bar{\bar{x}} \end{array}$$ Recall the following formulas $$\begin{array}{rcl} \text{SSE} &\stackrel{\text{(13.11)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2=\sum_{j=1}^{k} (n_j-1)s_j^2 \\ \text{SSTR} &\stackrel{\text{(13.8)}}{=}& \sum_{j=1}^{k} n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ \text{SST} &\stackrel{\text{(13.13)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \end{array}$$ Then $$\begin{array}{lll} \text{SST} &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j+\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}\left[(x_{ij}-\overline{x}_j)^2+2(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+(\overline{x}_j-\bar{\bar{x}})^2\right] \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\sum_{j=1}^{k}\sum_{i=1}^{n_j}(\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\sum_{j=1}^{k}n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \text{SSE}+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\text{SSTR} \\ \end{array}$$ We show that the middle term is zero. Indeed, $$\begin{array}{lll} \sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}}) &=& \sum_{i=1}^{n_j}x_{ij}\overline{x}_j-\sum_{i=1}^{n_j}\overline{x}_j^2-\sum_{i=1}^{n_j}x_{ij}\bar{\bar{x}}+\sum_{i=1}^{n_j}\overline{x}_j\bar{\bar{x}} \\ &=& n_j \overline{x}_j^2-n_j \overline{x}_j^2-n_j \overline{x}_j \bar{\bar{x}}+n_j \overline{x}_j \bar{\bar{x}} \\ &=& 0. \end{array}$$

(13.16) Fisher's LSD

$$t=\frac{\overline{x}_i-\overline{x}_j}{\sqrt{\text{MSE}\left(\frac{1}{n_i}+\frac{1}{n_j}\right)}}$$

Proof: See [Casella, sec.11.2].

(14.11) Relationship among SST, SSR, and SSE

$$\text{SST}=\text{SSR}+\text{SSE}$$

Proof: Recall that

$$\begin{array}{rcl} \text{SSE} &\stackrel{\text{(14.8)}}{=}& \sum(y_i-\hat{y}_i)^2 \\ \text{SST} &\stackrel{\text{(14.9)}}{=}& \sum(y_i-\overline{y})^2 \\ \text{SSR} &\stackrel{\text{(14.10)}}{=}& \sum(\hat{y}_i-\overline{y})^2. \end{array}$$ Note that $$\begin{array}{rcl} \text{SST} &=& \sum(y_i-\overline{y})^2 \\ &=& \sum(y_i-\hat{y}_i+\hat{y}_i-\overline{y})^2 \\ &=& \sum[(y_i-\hat{y}_i)^2+2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+(\hat{y}_i-\overline{y})^2] \\ &=& \sum(y_i-\hat{y}_i)^2+\sum 2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+\sum(\hat{y}_i-\overline{y})^2 \\ &=& \text{SSE}+\sum 2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+\text{SSR}. \end{array}$$ It suffices to show that $$\sum (y_i-\hat{y}_i)(\hat{y}_i-\overline{y})=0.$$ Recall the following formulas $$\begin{array}{rcl} \hat{y}_i &\stackrel{\text{(14.3)}}{=}& b_0+b_1 x_i \\ b_1 &\stackrel{\text{(14.6)}}{=}& \frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{\sum(x_i-\overline{x})^2} \\ b_0 &\stackrel{\text{(14.7)}}{=}& \overline{y}-b_1\overline{x}. \end{array}$$ Thus, $$\begin{array}{rcl} (\star)\to \sum (y_i-\hat{y}_i)(\hat{y}_i-\overline{y}) &\stackrel{\text{(14.3)}}{=}& \sum(y_i-b_0-b_1 x_i)(b_1 x_i+b_0-\overline{y}) \\ &\stackrel{\text{(14.7)}}{=}& \sum(y_i-\overline{y}+b_1\overline{x}-b_1 x_i)(b_1 x_i+\overline{y}-b_1 \overline{x}-\overline{y}) \\ &=& \sum\left[(y_i-\overline{y})-b_1(x_i-\overline{x})\right]\left[b_1(x_i-\overline{x})\right] \\ &=& b_1\sum(x_i-\overline{x})(y_i-\overline{y})-b_1^2\sum(x_i-\overline{x})^2 \\ &\stackrel{\text{(14.6)}}{=}& 0. \end{array}$$

(14.13) Sample Correlation Coefficient

$$\begin{array}{lll} r_{xy} &=& (\text{sign of }b_1)\sqrt{\text{Coefficient of determination}} \\ &=& (\text{sign of }b_1)\sqrt{r^2} \end{array}$$

Proof: Recall the following formulas

$$\begin{array}{rcl} s_x&\stackrel{\text{(3.8), (3.9)}}{=}& \sqrt{\frac{\sum(x_i-\overline{x})^2}{n-1}} \\ s_y&\stackrel{\text{(3.8), (3.9)}}{=}& \sqrt{\frac{\sum(y_i-\overline{y})^2}{n-1}} \\ s_{xy}&\stackrel{\text{(3.13)}}{=}& \frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{n-1} \\ r_{xy}&\stackrel{\text{(3.15)}}{=}& \frac{s_{xy}}{s_x s_y} \\ \text{SST} &\stackrel{\text{(14.9)}}{=}& \sum(y_i-\overline{y})^2 \\ \text{SSR} &\stackrel{\text{(14.10)}}{=}& \sum(\hat{y}_i-\overline{y})^2 \\ r^2 &\stackrel{\text{(14.12)}}{=}&\frac{\text{SSR}}{\text{SST}}. \end{array}$$ We express $\text{SSR}$ and $\text{SST}$ in term of $s_x, s_y$ and $s_{xy}$. It is easy to see that $$\text{SST}=(n-1)s_y^2.$$ In addition, $$\begin{array}{rcl} \text{SSR} &=& \sum(\hat{y}_i-\overline{y})^2 \\ &\stackrel{\text{See }(\star)\text{ in this webpage}}{=}& \sum(\hat{y}_i-\overline{y})^2+\sum(y_i-\hat{y}_i)(\hat{y}_i-\overline{y}) \\ &=& \sum(\hat{y}_i-\overline{y})[(\hat{y}_i-\overline{y})+(y_i-\hat{y}_i)] \\ &=& \sum(\hat{y}_i-\overline{y})(y_i-\overline{y}) \\ &\stackrel{\text{(14.3)}}{=}& \sum(b_0+b_1 x_i-\overline{y})(y_i-\overline{y}) \\ &\stackrel{\text{(14.7)}}{=}& \sum(b_1 x_i-b_1 \overline{x})(y_i-\overline{y}) \\ &=& b_1 \sum(x_i-\overline{x})(y_i-\overline{y}) \leftarrow (\star\star) \\ &\stackrel{\text{(14.6)}}{=}& \frac{\left[\sum(x_i-\overline{x})(y_i-\overline{y})\right]^2}{\sum(x_i-\overline{x})^2} \\ &=& \frac{(n-1)^2 s_{xy}^2}{(n-1)s_x^2}. \end{array}$$ Therefore, $$r^2 =\frac{\text{SSR}}{\text{SST}} =\frac{(n-1)^2 s_{xy}^2}{(n-1)s_x^2 (n-1)s_y^2} =\frac{s_{xy}^2}{s_x^2 s_y^2} =r_{xy}^2.$$ This follows that $$r_{xy}=\pm \sqrt{r^2}.$$ Note that $$b_1=\frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{\sum(x_i-\overline{x})^2}=\frac{(n-1)s_{xy}}{(n-1)s_x^2}=\frac{s_{xy}}{s_x^2}.$$ Which means that $b_1$ and $s_{xy}$ have the same sign. On the other hand, $s_x$ and $s_y$ are nonnegative. So $\frac{s_{xy}}{s_x s_y}$ and $b_1$ have the same sign. Then we have $$r_{xy}=(\text{sign of }b_1)\sqrt{r^2}.$$

(14.15) Mean Square Error (Estimate of $\sigma^2$)

$$s^2=\text{MSE}=\frac{\text{SSE}}{n-2}$$

Proof: See [Casella, p.552, (11.3.29).

(14.19) Test Statistic for t Test for Significance in Simple Linear Regression

$$t=\frac{b_1}{\hat{\sigma}_{b_1}}$$

Proof: By [Casella, p.553, thm.11.3.3], we have

$$b_1\sim \text{n}\left(\beta_1, \frac{\sigma^2}{\sum (x_i-\overline{x})^2}\right) \text{ and } \frac{\text{SSE}}{\sigma^2}\sim \chi^2(n-2).$$ This follows that $$\frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\sim \text{n}(0, 1)$$ and $$\frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\Big/\sqrt{\frac{\text{SSE}/\sigma^2}{n-2}} \stackrel{\hat{\sigma}=\sqrt{\frac{\text{SSE}}{n-2}}}{=}\frac{b_1-\beta_1}{\hat{\sigma}/\sqrt{\sum(x_1-\overline{x})^2}} =\frac{b_1-\beta_1}{\hat{\sigma}_{b_1}} \sim t(n-2)$$

(14.21) Test Statistic for F Test for Significance in Simple Linear Regression

$$F=\frac{\text{MSR}}{\text{MSE}}$$

Proof: By [Casella, p.553, thm.11.3.3], we have

$$b_1\sim \text{n}\left(\beta_1, \frac{\sigma^2}{\sum (x_i-\overline{x})^2}\right) \text{ and } \frac{\text{SSE}}{\sigma^2}\sim \chi^2(n-2).$$ Thus, $$\begin{array}{cl} & \frac{b_1-\beta_1}{\frac{\sigma}{\sqrt{\sum(x_i-\overline{x})^2}}}=\frac{(b_1-\beta_1)\sqrt{\sum(x_i-\overline{x})^2}}{\sigma}\sim \text{n}(0, 1) \\ \stackrel{\text{[Casella, p.53, exa.2.1.9]}}{\Rightarrow} & \frac{(b_1-\beta_1)^2\sum(x_i-\overline{x})^2}{\sigma^2}\sim \chi^2(1) \\ \stackrel{H_0:\beta_1=0}{\Rightarrow} & \frac{b_1^2\sum(x_i-\overline{x})^2}{\sigma^2}\sim \chi^2(1) \\ \stackrel{\text{(14.6)}}{\Rightarrow} & \frac{b_1 \sum(x_i-\overline{x})(y_i-\overline{y})}{\sigma^2}\sim \chi^2(1) \\ \stackrel{\text{See }(\star\star)\text{ in this webpage}}{\Rightarrow} & \frac{\text{SSR}}{\sigma^2}\sim \chi^2(1) \end{array}$$ Therefore, $$\frac{\text{MSR}}{\text{MSE}} =\frac{\text{SSR}/1}{\text{SSE}/(n-2)} =\frac{\frac{\text{SSR}}{\sigma^2}\big/ 1}{\frac{\text{SSE}}{\sigma^2}\big/(n-2)} \sim F(1, n-2).$$ We can also apply the square of a t distribution is a F distribution. See [Casella, p.225, thm.5.3.8]. That is, $$\begin{array}{rcl} \frac{\text{MSR}}{\text{MSE}} &=& \frac{\text{SSR}/1}{\text{SSE}/(n-2)} \\ &=& \frac{\frac{\text{SSR}}{\sigma^2}\big/ 1}{\frac{\text{SSE}}{\sigma^2}\big/(n-2)} \\ &=& \frac{b_1^2\sum(x_i-\overline{x})^2}{\sigma^2} \big/ \frac{\text{SSE}/\sigma^2}{n-2} \\ &=& \left(\frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\Big/\sqrt{\frac{\text{SSE}/\sigma^2}{n-2}}\right)^2 \\ &\stackrel{\text{(14.19)}}{\sim}& [t(n-2)]^2 \\ &=& F(1, n-2). \end{array}$$

(14.27) Prediction Interval for y*

$$\hat{y}^*\pm t_{\alpha/2}s_{\text{pred}} =\hat{y}^*\pm t_{\alpha/2}s\sqrt{1+\frac{1}{n}+\frac{(x^*-\overline{x})^2}{\sum(x_i-\overline{x})^2}} =\hat{y}^*\pm t_{\alpha/2}\sqrt{\frac{\text{SSE}}{n-2}}\sqrt{1+\frac{1}{n}+\frac{(x^*-\overline{x})^2}{\sum(x_i-\overline{x})^2}}$$ where the confidence coefficient is $1-\alpha$ and $t_{\alpha/2}$ is based on the $t$ distribution with $n-2$ degrees of freedom.

Proof: See [Casella, p.559, (11.3.41)].

(14.30) Standard Deviation of the ith Residual

$$s_{y_i-\hat{y}_i}=s\sqrt{1-h_i}, s\stackrel{\text{(14.16)}}{=}\sqrt{\frac{\text{SSE}}{n-2}}=\sqrt{\frac{\sum(y_i-\hat{y}_i)}{n-2}}, h_i=\frac{1}{n}+\frac{(x_i-\overline{x})^2}{\sum(x_i-\overline{x})^2}$$

Proof: By [Casella, p.552, (11.3.28),

$$\text{Var}(y_i-\hat{y}_i) =\left[ \frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right) \right]\sigma^2,$$ where $S_{xx}=\sum_{i=1}^{n}(x_i-\overline{x})^2$, see [Casella, p.541, (11.3.6). By [Anderson, p.667, (14.16)], $s$ is an estimator of $\sigma$, so we substitute $\sigma$ by $s$. Then we have $$s^2_{y_i-\hat{y}_i}=\left[ \frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right) \right]s^2$$ To prove $s_{y_i-\hat{y}_i}=s\sqrt{1-h_i}$, it suffices to check that $$1-h_i=1-\frac{1}{n}-\frac{(x_i-\overline{x})^2}{\sum(x_i-\overline{x})^2} =\frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right)$$

(14.33) Leverage of Observation i

$$h_i=\frac{1}{n}+\frac{(x_i-\bar{x})^2}{\sum(x_i-\bar{x})^2}$$

Proof: See Casella, subsec.11.3.5.

(15.9) Adjusted Multiple Coefficient of Determination

$$R_{\text{a}}^2=1-(1-R^2)\frac{n-1}{n-p-1}$$

Proof: 這個公式也可以表示成

$$R_{\text{a}}^2 =1-(1-R^2)\frac{n-1}{n-p-1} =1-\left(1-\frac{\text{SSR}}{\text{SST}}\right)\frac{n-1}{n-p-1} =1-\frac{\text{SSE}}{\text{SST}}\times\frac{n-1}{n-p-1}$$ Hastie在An Introduction to Statistical Learning中的解釋是（p.212, line -1)：The intuition behind the adjusted $R^2$ is that once all of the correct variables have been included in the model, adding additional noise variables will lead to only a very small decrease in $\text{RSS}$. Since adding noise variables leads to an increase in $d$, such variables will lead to an increase in $\frac{\text{RSS}}{n-d-1}$, and consequently a decrease in the adjusted $R^2$. Therefore, in theory, the model with the largest adjusted $R^2$ will have only correct variables and no noise variables. Unlike the $R^2$ statistic, the adjusted $R^2$ statistic pays a price for the inclusion of unnecessary variables in the model.

Hastie在這裡用的符號 $d$ 就是我們的 $p$，$\text{RSS}$ 就是我們的 $\text{SSE}$。

我的解釋是，如果加入更多的independent variables，則 $p$ 會變大，$\text{SSE}$ 會變小，這時有兩種情形：

• 如果 $\text{SSE}$ 只有變小一點點，那麼 $p$ 的變化對 $R_{\text{a}}^2$ 的影響比較大，就會讓 $R_{\text{a}}^2$ 變小，就表示加入這些independent variables是不好的；
• 如果 $\text{SSE}$ 變小很多，那麼 $\text{SSE}$ 的變化對 $R_{\text{a}}^2$ 的影響比較大，就會讓 $R_{\text{a}}^2$ 變大，就表示加入這些independent variables是好的。

(18.8) Spearman Rank-Correlation Coefficient

$$r_s=1-\frac{6\sum_{i=1}^{n}d_i^2}{n(n^2-1)}$$

課本有誤，誤寫成 $n^2+1$。

Proof: See [Hogg, IMS, p.634, subsec. 10.8.2]. It gives another more instructive expression

$$r_S=\frac{\sum\left[R(X_i)-\frac{n+1}{2}\right]\left[R(Y_i)-\frac{n+1}{2}\right]}{n(n^2-1)/12}$$ Note that $\frac{n+1}{2}$ and $\frac{n^2-1}{12}$ is the mean and variance of the discrete uniform distribution, respectively. Use $$\sum R(X_i)^2=\sum R(Y_i)^2=\frac{1}{6}n(n+1)(2n+1)$$ and $$\sum R(X_i)=\sum R(Y_i)=\frac{n(n+1)}{2}$$ to show that $$1-\frac{6\sum d_i^2}{n(n^2-1)} =1-\frac{6\sum [R(X_i)-R(Y_i)]^2}{n(n^2-1)}$$ and $$\frac{\sum\left[R(X_i)-\frac{n+1}{2}\right]\left[R(Y_i)-\frac{n+1}{2}\right]}{n(n^2-1)/12}$$ both are equal to $$\frac{12\sum R(X_i)R(Y_i)-3n(n+1)^2}{n(n^2-1)}$$

References

• [Casella] Casella and Berger's Statistical Inference
• [Hogg, PSI] Hogg and Tanis's Probability and Statistical Inference
• [Hogg, IMS] Hogg, McKean and Craig's Introduction to Mathematical Statistics