Problem 1 (10 points)

Let z be a 2 ◊ 1 random vector with distribution,

z ≥ N

350

0

6

,

5 1 ≠1

2

≠1

2 1

64 .

Find a 2 ◊ 2 matrix A such that ÎAzÎ2 has a chi-square distribution, and specify the associated degrees of

freedom.

Problem 2 (15 points) 3+3+3+6 = 15

Consider the following two regression models (at the population level):

M1 : y = —0 + —1×1 + —2×2 + Á,

M2 : y = —0 + —1×1 + —2×2 + —3×2

1 + —4×2

2 + —5x1x2 + Á.

Assume that we fit the two models to the same data.

(a) Can we compare the R2 of the two models without looking at the results of the fittings? Justify your

answer.

(b) Can we say that the residual vector of one model is orthogonal to the fitted value vector of the other

model? If so specify the details, if not argue why not.

(c) Assume that the data is actually generated from the following model (at the population level)

y = —ú

0 + —ú

1×1 + —ú

2×2 + —ú

3x1x2 + Á,

with —ú

0 , —ú

1 , —ú

2 and —ú

3 all nonzero. Which of the two models M1 and M2 will produce an unbiased

LSE of —ú? Justify your answer.

(d) Now assume the data of sample size n = 100 is actually generated from model M1 with noise vector

following our standard Gaussian assumption Á ≥ N(0, ‡2In). Consider an F-test for comparing M1

and M2. What is the probability that the p-value of that test is Ø 0.05? Justify your answer.

1

Problem 3 (20 points) 10+10 = 20

Consider the following two regression models (at the population level):

M1 : y = —0 + —1×1 + —2×2 + Á,

M2 : y = —0 + —1×1 + —2×2 + —3×2

1 + —4x1x2 + Á.

We fit the two models to the following data (n = 10):

t(dat)

## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]

## X1 -0.4 0.6 -0.2 0.8 0.9 -0.9 0.1 0.8 0.1 -0.1

## X2 0.9 -0.1 0.4 0.1 -0.8 0.8 -0.5 -0.9 -0.3 0.9

## y 2.0 -0.3 0.7 -0.4 -0.6 2.0 -0.4 -1.0 -0.2 0.9

(a) Compare the two models and choose one based on an F-test at level – = 0.05.

(b) Find the variance inflation factor for —1 in model M2.

Problem 4 (20 points)

Consider the following three regression models (at the population level):

M0 : y = —0 + Á,

M1 : y = —0 + —1×1 + —2×2 + Á,

M2 : y = —0 + —1×1 + —2×2 + —3×2

1 + —4×2

2 + —5x1x2 + Á.

We fit the three models to the same data with sample size n. For i = 0, 1, 2, let SSE(Mi) œ R and µˆ(Mi) œ Rn

denote the SSE and the fitted value vector of model Mi, respectively. The following information is given:

• The sample size n = 15.

• Îµˆ(M0) ≠ µˆ(M1)Î2 = 4 ◊ SSE(M1).

• The F-statistic for an F-test comparing models M1 and M2 is Fˆ = 4.

What is the R2 for model M2?

2

Problem 5 (15 points) 3 + 3 + 3 + 3 + 3 = 15

Consider regression model y = —0 + q4

j=1 —jxj + Á with the data given by

t(dat)

## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]

## X1 0.0 -1 0.0 -1 -1 0 1.0 -1.0 1 0

## X2 -1.0 0 1.0 1 0 -1 0.0 0.0 0 -1

## X3 -1.0 1 1.0 -1 1 1 1.0 1.0 0 0

## X4 -1.0 -1 1.0 -1 0 0 -1.0 0.0 0 0

## y -0.7 -1 -0.1 1 -1 -4 -0.8 -0.3 -3 2

Under the standard assumption Á ≥ N(0, ‡2In), answer the following:

(a) Which of the 10 data points has the highest influence on the regression, based on the Cook’s distance?

(b) Which of the 10 data points has the highest leverage without having much of an influence on the

regression?

(c) Using the PRESS statistic, among the the following two models, which one should be selected:

(1) the model that includes x1 and x2 or

(2) the model that includes x3 and x4?

Both models also include the intercept.

(d) Is it reasonable to use R2 to select among the two models in part (c)? Justify your answer.

(e) Argue that the comparison based on R2 in part (d) is equivalent to comparing the p-values of two

F-tests. Specify those tests.

Problem 6 (20 points) 10+10 = 20

Consider linear regression model y = —0 + q2

j=1 —jxj + Á with the transposed design matrix XT given by

## [,1] [,2] [,3] [,4] [,5]

## X1 0 -1 0 -1 -1

## X2 0 1 -1 1 0

and the transposed response vector yT given by

## [,1] [,2] [,3] [,4] [,5]

## y 3 -1 -4 -5 -3

The noise vector Á = [Á1,…, Án]

T follows the following model: (n = 5)

Á1 = z1

Ái ≠ Ái≠1 = zi, i = 2,…,n

zi ≥ N!

0, i

n

“

, i = 1, . . . , n.

Assume that {z1, z2,…,zn} are independent.

(a) What is the distribution of Á = [Á1, Á2,…, Án]

T ?

(b) What is the smallest possible variance of an unbiased estimate of —2?

Sample Solution