Eastern Nazarene College Multivariate Regression with R Code Answers

User Generated

axbfgnf

Mathematics

Eastern Nazarene College

Description

Please refer to the attached file.

Some of questions are required to writing down the codes to calculate some values; some other questions are analysis tasks

Please closely read assigned questions

Unformatted Attachment Preview

1 Homework 6: Multivariate Regression 1.1 Purpose Homework 6 is meant to give you some practice on understanding what can go wrong with multivariate regression. 1.2 What needs to be returned? • Please upload a typed out solution for the following questions to CourseWorks before class starts. 1.3 1.3.1 Math to Code Q1 Define a random vector with 3 random variables: • X ∼ N ormal(0, 10) • Y ∼ Exp(λ = 0.1) • Z = Y + 2 ∗ X + , where  ∼ U nif [−5, 5] Please assume that X, Y , and  are all independent  one another.  from X Please calculate the theoretical values of Cov  Y , hint: it’s 3 by 3 Z matrix. 1.3.2 Q2, numerically approximating covariances Please test out 2 sample sizes, 100 and 10000 to numerically approximate the 3 by 3 covariance matrix from Q1 via simulation. You should use the sample covariance to approximate the theoretical covariance matrix, i.e. Σ̂n ≈ Σ. Σ̂n is the sample covariance based on sample size n. Create 200 simulations for each sample size to approximate the covariance matrix above (i.e. you would have 200 covariance estimates for each sample size). If the theoretical covariance is Σ and the estimate is Σ̂, define the error qPmatrix P3 3 2 as kΣ − Σ̂k2 = kDk2 = i=1 j=1 Di,j ). This is called the Frobenius norm for the matrix D. Please report the 2.5 to 97.5 percentile values of the Frobenius norm, across the 200 simulations for each sample size. Please comment on the sample size’s effect on the accuracy of the numerical approximation of Σ. 1 1.3.3 Q3, a common abuse of the word ”sample size” In this question, the word ”sample size” is used in 2 different ways that is common and confusing. In the regression setting, we often describe Y = Xβ + , where Cov(|X) = σ 2 I as a n × n matrix. This σ 2 I is the theoretical covariance. In the case where n = 20, i.e. the sample size of the regression is 20, please write the code that would numerically approximate the covariance matrix σ 2 I using the sample size with the smaller error from Q2 by simulating different  vectors. Please set σ 2 = 4. I’m intentionally not prescribing the distribution of , choose your favorite distribution. :) 1.3.4 Q4 Let there be 20 samples. Let X1 ∼ Bernoulli(0.3), let X2 ∼ U nif [−10, 10], let X3 ∼ N ormal(100, 10), let X4 = 2 ∗ X1 − X2 + 0.3X3, let X5 = 1 − X1, and finally define X0 as the constant feature of 1’s. Please examine the eigen values for the matrix X T X with the following definitions of X and report whether (X T X)−1 exists. Note, the notation below indicates combining the vectors by columns   • X = X0 : X1 : X2 : X3 : X4 : X5   • X = X1 : X2 : X3 : X4 : X5   • X = X1 : X2 : X3 : X5   • X = X0 : X2 : X3 : X4 : X5   • X = X2 : X3 : X4 : X5   • X = X0 : X1 : X2 : X3 : X4   • X = X0 : X1 : X5 1.4 1.4.1 Simultaneous Inference Q5 Imagine Z ∼ Binomial(n = 100, p = 0.05), please report an 95% prediction interval for Z. Note, by convention, prediction intervals centers the expected value, but this is not technically required. A 95% prediction interval is any interval such that, when predicting the value of Z, will have a 95% chance of containing Z. 2 1.4.2 Q6 Let the sample size be 1000, Y ∼ N ormal(0, 10) then create 99 random features that are completely uncorrelated with Y . Please regress Y on these features and report the number of significant features using point wise hypothesis tests, i.e. | SE β̂i (β̂ | ≥ t(n − p, 97.5) would identify a significant feature. i σ̂ 2 rh i Recall that SEσ̂2 (β̂i ) = Cov(β̂|X) where we use σ̂ 2 to approximate i,i σ 2 . Since you have the intercept, there should be a total of 100 tests being performed. 1.4.3 Q7 Continuing from Q6, let us adjusted the problem by using the Bonferroni correction to perform simultaneous inference. Please write the code that would numerically show that the false positive rate from Bonferroni is at most 5% over 1000 simulations. In other words, the probability of calling at least one feature significant when all coefficients are 0, is upper bounded by 5%. 1.5 1.5.1 Interpreting your model Q8 Let the sample size be 200. Define X ∼ N ormal(10, 10), let Z ∼ Bernoulli(0.4), and let Y = 5 − X + 2 ∗ Z + 3 ∗ X ∗ Z +  where  ∼ U nif [−3, 3]. Please run the regression using all the data that includes the interaction effect and report the coefficients, let’s call these β̂. You should have 4 coefficients. Side note, this is an example of how you can imagine data to be generated from different groups that have different intercepts and slopes. 1.5.2 Q9 Please regress Y on X only using the values where Z = 0. Repeat this regression only using the values where Z = 1. Report those coefficients, let’s call them β̂0 and β̂1 respectively. 1.5.3 Q10 Assume the parameters below refer to the coefficients from β̂, β̂0 and β̂1 . Please answer the following: • The intercept for β̂ equals which other parameter? • The intercept for β̂1 is the sum between which 2 parameters? • The slope for β̂0 is the same as which other parameter? • The slope for β̂1 equals to the linear combination of which other parameters? 3 1.5.4 Q11 Q8-Q10 shows a case where we can obtain identical regression estimates by regressing with interactions or by training 2 separate regression models, are the standard errors for these estimates the same, yes/no? A thought you should have: ”which method would you choose if someone asked you to choose?” (No need to answer this question for Q11). 4
Purchase answer to see full attachment
User generated content is uploaded by users for the purposes of learning and should be used following Studypool's honor code & terms of service.

Explanation & Answer

Hi, here is your assignment, kindly see attached ;)

1
1 Homework 6: Multivariate Regression
1.1 Purpose
Homework 6 is meant to give you some practice on understanding what can go wrong with multivariate
regression.
1.2 What needs to be returned?
• Please upload a typed out solution for the following questions to CourseWorks before class starts.
1.3 Math to Code
1.3.1 Q1
Define a random vector with 3 random variables:
R code solution:


X ~ Normal(0, 10)

X ~ Normal(0, 10)
> X X
[1] 1.1901502 1.2200305 0.3301327 -1.2165371 -1.3769804 -0.8329006
[7] 1.0076789 0.6001681 -0.5144466 -0.1612686


Y ∼ Exp(λ = 0.1)

> Y = exp(0.1)
> Y
[1] 1.105171



Z = Y + 2 ∗ X + ϵ, where ϵ ∼ Unif[−5, 5]

> ϵ ϵ
[1] 3.412053
> Z=y Z
[1] 6.897524 6.957285 5.177489 2.084149 1.763263 2.851422 6.532581 5.717560
[9] 3.488330 4.194686

Definition of vectors: It is given that X, Y, Z have normal distribution, both are independent.
Matrix:
> (xy Z cov.wt(xy, wt = w1) # i.e. method = "unbiased"
$cov
X Y
X 1.195985 0
Y 0.000000 0
$center
X
-0.3637142

Y
1.1051709

$n.obs
[1] 10
$wt
[1] 0.0 0.0 0.0 0.2 0.2 0.2 0.2 0.2 0.0 0.0
> cov.wt(xy, wt = w1, method = "ML", cor = TRUE)
$cov
X Y
X 0.9567879 0
Y 0.0000000 0
$center
X
-0.3637142

Y
1.1051709

$n.obs
[1] 10
$wt
[1] 0.0 0.0 0.0 0.2 0.2 0.2 0.2 0.2 0.0 0.0
$cor
X
Y
X
1 NaN
Y NaN NaN

3

1.3.2 Q2, numerically approximating covariances:
> require(ggplot2)
> sampa=rnorm(100,0,10)
> sampb=rnorm(1000,0,10)
> combined = c(sampa, sampb)
> plt = ggplot(data.frame(combined), aes(x=combined)) +
stat_bin(binwidth=0.25, position="identity")
> plt

> trans=matrix(c(1.2, 1.2, 0.33, -1.38, -0.83, 1.01, 0.60, -0.51, -0.16,
+
1.50),nrow=4)
> dimnames = list(c("A", "C", "G", "T"), c("A", "C", "G", "T"))
> trans
[,1] [,2] [,3]
[1,] 1.20 -0.83 -0.16
[2,] 1.20 1.01 1.50
[3,] 0.33 0.60 1.20
[4,] -1.38 -0.51 1.20

4

1.3.3
Q3, a common abuse of the word ”sample size”
> # Generating data for 20 sample of size 20
>
> sdata
>
>
>
>
>
>
>
>
+
+
+
>
>
>
>
>
>
>
>

crit X5 = 1 − X1
> X5
[1] 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1 1 0 0

X0 defines the y function that would be equal to 0.
> eigen(cbind(X1, X2, X3, X4), only.values = TRUE)
> eigen(cbind(-1, 2:1)) # complex values
eigen() decomposition
$values
[1] 0+1i 0-1i
$vectors
[,1]
[,2]
[1,] 0.8164966+0.0000000i 0.8164966+0.0000000i
[2,] 0.4082483+0.4082483i 0.4082483-0.4082483i
> eigen(print(cbind(c(0, 1i), c(-1i, 0)))) # Hermite ==> real Eigenvalues
[,1] [,2]
[1,] 0+0i 0-1i
[2,] 0+1i 0+0i
eigen()...


Anonymous
Just the thing I needed, saved me a lot of time.

Studypool
4.7
Trustpilot
4.5
Sitejabber
4.4

Related Tags