Lesson 17 - Adjusted R-Squared

R-Squared vs. Adjusted R-Squared

In this lesson, we will demonstrate the differences between the standard r-squared diagnostic measure and the adjusted r-squared meausure.

We begin by randomly generating 800 observations of a respnonse variable y as well as 600 potential predictors. The values for each variable are drawn from pairwise independent normal distributions.

n = 800
iter = 600
df <- data.frame(replicate(iter, rnorm(n, 10, 2)))
y <- rnorm(n, 10, 2)

To confirm that the variables are independent of one another, we will create a pairs plot of y and the first 4 predictors.

temp_df <- data.frame(y, df[1:4])
pairs(temp_df)

We will now create 600 regression models. Each model will use y as the response. Model number i will use the first i possible predictors. For each model, we will store the r-squared value, as well as the adjusted r-squared value.

r_sqr <- c()
adj_r <- c()
for (i in 1:iter){
  mod <- lm(y ~ ., df[1:i])
  s <- summary(mod)
  r_sqr <- c(r_sqr, s$r.squared)
  adj_r <- c(adj_r, s$adj.r.squared)
}

We will now create line plots of see how r-squared and adjusted r-squared change as we add in more predictors.

plot(r_sqr, pch=".", col="white", ylim = c(-0.2, 01))
abline(h=0)

lines(1:iter, r_sqr, lwd=2, col="salmon")
lines(1:iter, adj_r, lwd=2, col="cornflowerblue")