Proofs of some formulas in Anderson's Statistics for Business and Economics

• (9.7) Sample Size for a One-Tailed Hypothesis Test about a Population Mean
• (10.7) Degree of Freedom: t Distribution with Two Independent Random Samples
• (13.12) Test Statistic for the Equality of k Population Means
• (13.14) SSE+SSTR=SST
• (13.16) Fisher's LSD
• (14.11) Relationship among SST, SSR, and SSE
• (14.13) Sample Correlation Coefficient
• (14.15) Mean Square Error (Estimate of \(\sigma^2\))
• (14.19) Test Statistic for t Test for Significance in Simple Linear Regression
• (14.21) Test Statistic for F Test for Significance in Simple Linear Regression
• (14.27) Prediction Interval for y*
• (14.30) Standard Deviation of the ith Residual
• (14.33) Leverage of Observation of i
• (15.9) Adjusted Multiple Coefficient of Determination
• (18.8) Spearman Rank-Correlation Coefficient
• References

(9.7) Sample Size for a One-Tailed Hypothesis Test about a Population Mean

\[ n=\frac{(z_{\alpha}+z_{\beta})^2 \sigma^2}{(\mu_0-\mu_a)^2} \] In a two-tailed hypothesis test, use (9.7) with \(z_{\alpha/2}\) replacing \(z_{\alpha}\).

Proof: Note that we don't have to change \(\beta\) in a two-tailed hypothesis.

(10.7) Degree of Freedom: t Distribution with Two Independent Random Samples

\[ \text{d.f.}=\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{1}{n_1-1}\left(\frac{s_1^2}{n_1}\right)^2+\frac{1}{n_2-1}\left(\frac{s_2^2}{n_2}\right)^2} \]

Proof: See [Casella, p.409, exe.8.42].

(13.12) Test Statistic for the Equality of k Population Means

\[ F=\frac{\text{MSTR}}{\text{MSE}} \]

Proof: This proof is from [Hogg, PSI, sec.9.3]. Recall that \[ \begin{array}{cccccc} \text{treatment 1} & \text{treatment 2} & \cdots & \text{treatment k} \\ x_{11} & x_{12} & \cdots & x_{1k} \\ x_{21} & x_{22} & \cdots & x_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n_1 1} & x_{n_2 2} & \cdots & x_{n_k k} \\ \downarrow & \downarrow & \cdots & \downarrow & \searrow \\ \overline{x}_1 & \overline{x}_2 & \cdots & \overline{x}_k & & \bar{\bar{x}} \end{array} \] Recall the following formulas \[ \begin{array}{rcl} \text{SSE} &\stackrel{\text{(13.11)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2=\sum_{j=1}^{k} (n_j-1)s_j^2 \\ \text{SSTR} &\stackrel{\text{(13.8)}}{=}& \sum_{j=1}^{k} n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ \text{SST} &\stackrel{\text{(13.13)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \\ \text{MSE} &\stackrel{\text{(13.10)}}{=}& \frac{\text{SSE}}{n-k} \\ \text{MSTR} &\stackrel{\text{(13.7)}}{=}& \frac{\text{SSTR}}{k-1} \\ \end{array} \] At first, \[ \begin{array}{ll} & \frac{\text{SST}}{n-1}=\frac{\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2}{n-1}\stackrel{\text{用全部來算sample variance}}{=}S^2 \\ \stackrel{\text{[Casella, p.218, thm.5.3.1.(c)]}}{\Rightarrow} &\frac{\text{SST}}{\sigma^2}=\frac{(n-1)S^2}{\sigma^2}\sim \chi^2_{n-1} \end{array} \] Then \[ \begin{array}{cl} & \frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{n_j-1}\stackrel{\text{用單行來算sample variance}}{=}S^2 \\ \stackrel{\text{[Casella, p.218, thm.5.3.1.(c)]}}{\Rightarrow}& \frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{\sigma^2}=\frac{(n_j-1)S^2}{\sigma^2}\sim \chi^2_{n_j-1} \\ \stackrel{\text{[Casella, p.219, lem.5.3.2.(b)]}}{\Rightarrow}& \frac{\text{SSE}}{\sigma^2}=\sum_{j=1}^{k}\frac{\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2}{\sigma^2}\sim \chi^2_{(n_1-1)+(n_2-1)+\cdots +(n_k-1)}=\chi^2_{n-k} \\ \stackrel{\text{[Casella, p.155, thm.4.2.12]}}{\Rightarrow}& \frac{\text{SSTR}}{\sigma^2}=\frac{\text{SST}}{\sigma^2}-\frac{\text{SSE}}{\sigma^2}\sim \chi^2_{k-1} \\ \Rightarrow & \frac{\text{MSTR}}{\text{MSE}}=\frac{\text{SSTR}/(k-1)}{\text{SSE}/(n-k)}=\frac{(\text{SSTR}/\sigma^2)/(k-1)}{(\text{SSE}/\sigma^2)/(n-k)}\sim F(k-1, n-k) \end{array} \]

(13.14) SST=SSTR+SSE

\[ \text{SST}=\text{SSTR}+\text{SSE} \]

Proof: Recall that \[ \begin{array}{cccccc} \text{treatment 1} & \text{treatment 2} & \cdots & \text{treatment k} \\ x_{11} & x_{12} & \cdots & x_{1k} \\ x_{21} & x_{22} & \cdots & x_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n_1 1} & x_{n_2 2} & \cdots & x_{n_k k} \\ \downarrow & \downarrow & \cdots & \downarrow & \searrow \\ \overline{x}_1 & \overline{x}_2 & \cdots & \overline{x}_k & & \bar{\bar{x}} \end{array} \] Recall the following formulas \[ \begin{array}{rcl} \text{SSE} &\stackrel{\text{(13.11)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2=\sum_{j=1}^{k} (n_j-1)s_j^2 \\ \text{SSTR} &\stackrel{\text{(13.8)}}{=}& \sum_{j=1}^{k} n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ \text{SST} &\stackrel{\text{(13.13)}}{=}& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \end{array} \] Then \[ \begin{array}{lll} \text{SST} &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j+\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}\left[(x_{ij}-\overline{x}_j)^2+2(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+(\overline{x}_j-\bar{\bar{x}})^2\right] \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\sum_{j=1}^{k}\sum_{i=1}^{n_j}(\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)^2+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\sum_{j=1}^{k}n_j(\overline{x}_j-\bar{\bar{x}})^2 \\ &=& \text{SSE}+2\sum_{j=1}^{k}\sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}})+\text{SSTR} \\ \end{array} \] We show that the middle term is zero. Indeed, \[ \begin{array}{lll} \sum_{i=1}^{n_j}(x_{ij}-\overline{x}_j)(\overline{x}_j-\bar{\bar{x}}) &=& \sum_{i=1}^{n_j}x_{ij}\overline{x}_j-\sum_{i=1}^{n_j}\overline{x}_j^2-\sum_{i=1}^{n_j}x_{ij}\bar{\bar{x}}+\sum_{i=1}^{n_j}\overline{x}_j\bar{\bar{x}} \\ &=& n_j \overline{x}_j^2-n_j \overline{x}_j^2-n_j \overline{x}_j \bar{\bar{x}}+n_j \overline{x}_j \bar{\bar{x}} \\ &=& 0. \end{array} \]

(13.16) Fisher's LSD

\[ t=\frac{\overline{x}_i-\overline{x}_j}{\sqrt{\text{MSE}\left(\frac{1}{n_i}+\frac{1}{n_j}\right)}} \]

Proof: See [Casella, sec.11.2].

(14.11) Relationship among SST, SSR, and SSE

\[ \text{SST}=\text{SSR}+\text{SSE} \]

Proof: Recall that \[ \begin{array}{rcl} \text{SSE} &\stackrel{\text{(14.8)}}{=}& \sum(y_i-\hat{y}_i)^2 \\ \text{SST} &\stackrel{\text{(14.9)}}{=}& \sum(y_i-\overline{y})^2 \\ \text{SSR} &\stackrel{\text{(14.10)}}{=}& \sum(\hat{y}_i-\overline{y})^2. \end{array} \] Note that \[ \begin{array}{rcl} \text{SST} &=& \sum(y_i-\overline{y})^2 \\ &=& \sum(y_i-\hat{y}_i+\hat{y}_i-\overline{y})^2 \\ &=& \sum[(y_i-\hat{y}_i)^2+2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+(\hat{y}_i-\overline{y})^2] \\ &=& \sum(y_i-\hat{y}_i)^2+\sum 2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+\sum(\hat{y}_i-\overline{y})^2 \\ &=& \text{SSE}+\sum 2(y_i-\hat{y}_i)(\hat{y}_i-\overline{y})+\text{SSR}. \end{array} \] It suffices to show that \[ \sum (y_i-\hat{y}_i)(\hat{y}_i-\overline{y})=0. \] Recall the following formulas \[ \begin{array}{rcl} \hat{y}_i &\stackrel{\text{(14.3)}}{=}& b_0+b_1 x_i \\ b_1 &\stackrel{\text{(14.6)}}{=}& \frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{\sum(x_i-\overline{x})^2} \\ b_0 &\stackrel{\text{(14.7)}}{=}& \overline{y}-b_1\overline{x}. \end{array} \] Thus, \[ \begin{array}{rcl} (\star)\to \sum (y_i-\hat{y}_i)(\hat{y}_i-\overline{y}) &\stackrel{\text{(14.3)}}{=}& \sum(y_i-b_0-b_1 x_i)(b_1 x_i+b_0-\overline{y}) \\ &\stackrel{\text{(14.7)}}{=}& \sum(y_i-\overline{y}+b_1\overline{x}-b_1 x_i)(b_1 x_i+\overline{y}-b_1 \overline{x}-\overline{y}) \\ &=& \sum\left[(y_i-\overline{y})-b_1(x_i-\overline{x})\right]\left[b_1(x_i-\overline{x})\right] \\ &=& b_1\sum(x_i-\overline{x})(y_i-\overline{y})-b_1^2\sum(x_i-\overline{x})^2 \\ &\stackrel{\text{(14.6)}}{=}& 0. \end{array} \]

(14.13) Sample Correlation Coefficient

\[ \begin{array}{lll} r_{xy} &=& (\text{sign of }b_1)\sqrt{\text{Coefficient of determination}} \\ &=& (\text{sign of }b_1)\sqrt{r^2} \end{array} \]

Proof: Recall the following formulas \[ \begin{array}{rcl} s_x&\stackrel{\text{(3.8), (3.9)}}{=}& \sqrt{\frac{\sum(x_i-\overline{x})^2}{n-1}} \\ s_y&\stackrel{\text{(3.8), (3.9)}}{=}& \sqrt{\frac{\sum(y_i-\overline{y})^2}{n-1}} \\ s_{xy}&\stackrel{\text{(3.13)}}{=}& \frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{n-1} \\ r_{xy}&\stackrel{\text{(3.15)}}{=}& \frac{s_{xy}}{s_x s_y} \\ \text{SST} &\stackrel{\text{(14.9)}}{=}& \sum(y_i-\overline{y})^2 \\ \text{SSR} &\stackrel{\text{(14.10)}}{=}& \sum(\hat{y}_i-\overline{y})^2 \\ r^2 &\stackrel{\text{(14.12)}}{=}&\frac{\text{SSR}}{\text{SST}}. \end{array} \] We express \(\text{SSR}\) and \(\text{SST}\) in term of \(s_x, s_y\) and \(s_{xy}\). It is easy to see that \[ \text{SST}=(n-1)s_y^2. \] In addition, \[ \begin{array}{rcl} \text{SSR} &=& \sum(\hat{y}_i-\overline{y})^2 \\ &\stackrel{\text{See }(\star)\text{ in this webpage}}{=}& \sum(\hat{y}_i-\overline{y})^2+\sum(y_i-\hat{y}_i)(\hat{y}_i-\overline{y}) \\ &=& \sum(\hat{y}_i-\overline{y})[(\hat{y}_i-\overline{y})+(y_i-\hat{y}_i)] \\ &=& \sum(\hat{y}_i-\overline{y})(y_i-\overline{y}) \\ &\stackrel{\text{(14.3)}}{=}& \sum(b_0+b_1 x_i-\overline{y})(y_i-\overline{y}) \\ &\stackrel{\text{(14.7)}}{=}& \sum(b_1 x_i-b_1 \overline{x})(y_i-\overline{y}) \\ &=& b_1 \sum(x_i-\overline{x})(y_i-\overline{y}) \leftarrow (\star\star) \\ &\stackrel{\text{(14.6)}}{=}& \frac{\left[\sum(x_i-\overline{x})(y_i-\overline{y})\right]^2}{\sum(x_i-\overline{x})^2} \\ &=& \frac{(n-1)^2 s_{xy}^2}{(n-1)s_x^2}. \end{array} \] Therefore, \[ r^2 =\frac{\text{SSR}}{\text{SST}} =\frac{(n-1)^2 s_{xy}^2}{(n-1)s_x^2 (n-1)s_y^2} =\frac{s_{xy}^2}{s_x^2 s_y^2} =r_{xy}^2. \] This follows that \[ r_{xy}=\pm \sqrt{r^2}. \] Note that \[ b_1=\frac{\sum(x_i-\overline{x})(y_i-\overline{y})}{\sum(x_i-\overline{x})^2}=\frac{(n-1)s_{xy}}{(n-1)s_x^2}=\frac{s_{xy}}{s_x^2}. \] Which means that \(b_1\) and \(s_{xy}\) have the same sign. On the other hand, \(s_x\) and \(s_y\) are nonnegative. So \(\frac{s_{xy}}{s_x s_y}\) and \(b_1\) have the same sign. Then we have \[ r_{xy}=(\text{sign of }b_1)\sqrt{r^2}. \]

(14.15) Mean Square Error (Estimate of \(\sigma^2\))

\[ s^2=\text{MSE}=\frac{\text{SSE}}{n-2} \]

Proof: See [Casella, p.552, (11.3.29).

(14.19) Test Statistic for t Test for Significance in Simple Linear Regression

\[ t=\frac{b_1}{\hat{\sigma}_{b_1}} \]

Proof: By [Casella, p.553, thm.11.3.3], we have \[ b_1\sim \text{n}\left(\beta_1, \frac{\sigma^2}{\sum (x_i-\overline{x})^2}\right) \text{ and } \frac{\text{SSE}}{\sigma^2}\sim \chi^2(n-2). \] This follows that \[ \frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\sim \text{n}(0, 1) \] and \[ \frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\Big/\sqrt{\frac{\text{SSE}/\sigma^2}{n-2}} \stackrel{\hat{\sigma}=\sqrt{\frac{\text{SSE}}{n-2}}}{=}\frac{b_1-\beta_1}{\hat{\sigma}/\sqrt{\sum(x_1-\overline{x})^2}} =\frac{b_1-\beta_1}{\hat{\sigma}_{b_1}} \sim t(n-2) \]

(14.21) Test Statistic for F Test for Significance in Simple Linear Regression

\[ F=\frac{\text{MSR}}{\text{MSE}} \]

Proof: By [Casella, p.553, thm.11.3.3], we have \[ b_1\sim \text{n}\left(\beta_1, \frac{\sigma^2}{\sum (x_i-\overline{x})^2}\right) \text{ and } \frac{\text{SSE}}{\sigma^2}\sim \chi^2(n-2). \] Thus, \[ \begin{array}{cl} & \frac{b_1-\beta_1}{\frac{\sigma}{\sqrt{\sum(x_i-\overline{x})^2}}}=\frac{(b_1-\beta_1)\sqrt{\sum(x_i-\overline{x})^2}}{\sigma}\sim \text{n}(0, 1) \\ \stackrel{\text{[Casella, p.53, exa.2.1.9]}}{\Rightarrow} & \frac{(b_1-\beta_1)^2\sum(x_i-\overline{x})^2}{\sigma^2}\sim \chi^2(1) \\ \stackrel{H_0:\beta_1=0}{\Rightarrow} & \frac{b_1^2\sum(x_i-\overline{x})^2}{\sigma^2}\sim \chi^2(1) \\ \stackrel{\text{(14.6)}}{\Rightarrow} & \frac{b_1 \sum(x_i-\overline{x})(y_i-\overline{y})}{\sigma^2}\sim \chi^2(1) \\ \stackrel{\text{See }(\star\star)\text{ in this webpage}}{\Rightarrow} & \frac{\text{SSR}}{\sigma^2}\sim \chi^2(1) \end{array} \] Therefore, \[ \frac{\text{MSR}}{\text{MSE}} =\frac{\text{SSR}/1}{\text{SSE}/(n-2)} =\frac{\frac{\text{SSR}}{\sigma^2}\big/ 1}{\frac{\text{SSE}}{\sigma^2}\big/(n-2)} \sim F(1, n-2). \] We can also apply the square of a t distribution is a F distribution. See [Casella, p.225, thm.5.3.8]. That is, \[ \begin{array}{rcl} \frac{\text{MSR}}{\text{MSE}} &=& \frac{\text{SSR}/1}{\text{SSE}/(n-2)} \\ &=& \frac{\frac{\text{SSR}}{\sigma^2}\big/ 1}{\frac{\text{SSE}}{\sigma^2}\big/(n-2)} \\ &=& \frac{b_1^2\sum(x_i-\overline{x})^2}{\sigma^2} \big/ \frac{\text{SSE}/\sigma^2}{n-2} \\ &=& \left(\frac{b_1-\beta_1}{\sigma/\sqrt{\sum(x_i-\overline{x})^2}}\Big/\sqrt{\frac{\text{SSE}/\sigma^2}{n-2}}\right)^2 \\ &\stackrel{\text{(14.19)}}{\sim}& [t(n-2)]^2 \\ &=& F(1, n-2). \end{array} \]

(14.27) Prediction Interval for y*

\[ \hat{y}^*\pm t_{\alpha/2}s_{\text{pred}} =\hat{y}^*\pm t_{\alpha/2}s\sqrt{1+\frac{1}{n}+\frac{(x^*-\overline{x})^2}{\sum(x_i-\overline{x})^2}} =\hat{y}^*\pm t_{\alpha/2}\sqrt{\frac{\text{SSE}}{n-2}}\sqrt{1+\frac{1}{n}+\frac{(x^*-\overline{x})^2}{\sum(x_i-\overline{x})^2}} \] where the confidence coefficient is \(1-\alpha\) and \(t_{\alpha/2}\) is based on the \(t\) distribution with \(n-2\) degrees of freedom.

Proof: See [Casella, p.559, (11.3.41)].

(14.30) Standard Deviation of the ith Residual

\[ s_{y_i-\hat{y}_i}=s\sqrt{1-h_i}, s\stackrel{\text{(14.16)}}{=}\sqrt{\frac{\text{SSE}}{n-2}}=\sqrt{\frac{\sum(y_i-\hat{y}_i)}{n-2}}, h_i=\frac{1}{n}+\frac{(x_i-\overline{x})^2}{\sum(x_i-\overline{x})^2} \]

Proof: By [Casella, p.552, (11.3.28), \[ \text{Var}(y_i-\hat{y}_i) =\left[ \frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right) \right]\sigma^2, \] where \(S_{xx}=\sum_{i=1}^{n}(x_i-\overline{x})^2\), see [Casella, p.541, (11.3.6). By [Anderson, p.667, (14.16)], \(s\) is an estimator of \(\sigma\), so we substitute \(\sigma\) by \(s\). Then we have \[ s^2_{y_i-\hat{y}_i}=\left[ \frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right) \right]s^2 \] To prove \(s_{y_i-\hat{y}_i}=s\sqrt{1-h_i}\), it suffices to check that \[ 1-h_i=1-\frac{1}{n}-\frac{(x_i-\overline{x})^2}{\sum(x_i-\overline{x})^2} =\frac{n-2}{n}+\frac{1}{S_{xx}}\left(\frac{1}{n}\sum_{j=1}^{n}x_j^2+x_i^2-2(x_i-\overline{x})^2-2x_i\overline{x}\right) \]

(14.33) Leverage of Observation i

\[ h_i=\frac{1}{n}+\frac{(x_i-\bar{x})^2}{\sum(x_i-\bar{x})^2} \]

Proof: See Casella, subsec.11.3.5.

(15.9) Adjusted Multiple Coefficient of Determination

\[ R_{\text{a}}^2=1-(1-R^2)\frac{n-1}{n-p-1} \]

Proof: 這個公式也可以表示成 \[ R_{\text{a}}^2 =1-(1-R^2)\frac{n-1}{n-p-1} =1-\left(1-\frac{\text{SSR}}{\text{SST}}\right)\frac{n-1}{n-p-1} =1-\frac{\text{SSE}}{\text{SST}}\times\frac{n-1}{n-p-1} \] Hastie在An Introduction to Statistical Learning中的解釋是（p.212, line -1)：The intuition behind the adjusted \(R^2\) is that once all of the correct variables have been included in the model, adding additional noise variables will lead to only a very small decrease in \(\text{RSS}\). Since adding noise variables leads to an increase in \(d\), such variables will lead to an increase in \(\frac{\text{RSS}}{n-d-1}\), and consequently a decrease in the adjusted \(R^2\). Therefore, in theory, the model with the largest adjusted \(R^2\) will have only correct variables and no noise variables. Unlike the \(R^2\) statistic, the adjusted \(R^2\) statistic pays a price for the inclusion of unnecessary variables in the model.

Hastie在這裡用的符號 \(d\) 就是我們的 \(p\)，\(\text{RSS}\) 就是我們的 \(\text{SSE}\)。

我的解釋是，如果加入更多的independent variables，則 \(p\) 會變大，\(\text{SSE}\) 會變小，這時有兩種情形：

• 如果 \(\text{SSE}\) 只有變小一點點，那麼 \(p\) 的變化對 \(R_{\text{a}}^2\) 的影響比較大，就會讓 \(R_{\text{a}}^2\) 變小，就表示加入這些independent variables是不好的；
• 如果 \(\text{SSE}\) 變小很多，那麼 \(\text{SSE}\) 的變化對 \(R_{\text{a}}^2\) 的影響比較大，就會讓 \(R_{\text{a}}^2\) 變大，就表示加入這些independent variables是好的。

(18.8) Spearman Rank-Correlation Coefficient

\[ r_s=1-\frac{6\sum_{i=1}^{n}d_i^2}{n(n^2-1)} \]

課本有誤，誤寫成 \(n^2+1\)。

Proof: See [Hogg, IMS, p.634, subsec. 10.8.2]. It gives another more instructive expression \[ r_S=\frac{\sum\left[R(X_i)-\frac{n+1}{2}\right]\left[R(Y_i)-\frac{n+1}{2}\right]}{n(n^2-1)/12} \] Note that \(\frac{n+1}{2}\) and \(\frac{n^2-1}{12}\) is the mean and variance of the discrete uniform distribution, respectively. Use \[ \sum R(X_i)^2=\sum R(Y_i)^2=\frac{1}{6}n(n+1)(2n+1) \] and \[ \sum R(X_i)=\sum R(Y_i)=\frac{n(n+1)}{2} \] to show that \[ 1-\frac{6\sum d_i^2}{n(n^2-1)} =1-\frac{6\sum [R(X_i)-R(Y_i)]^2}{n(n^2-1)} \] and \[ \frac{\sum\left[R(X_i)-\frac{n+1}{2}\right]\left[R(Y_i)-\frac{n+1}{2}\right]}{n(n^2-1)/12} \] both are equal to \[ \frac{12\sum R(X_i)R(Y_i)-3n(n+1)^2}{n(n^2-1)} \]

References

• [Casella] Casella and Berger's Statistical Inference
• [Hogg, PSI] Hogg and Tanis's Probability and Statistical Inference
• [Hogg, IMS] Hogg, McKean and Craig's Introduction to Mathematical Statistics