---
title: "Simple linear regression (Notes)"
subtitle: "STAT 155"
author: "Your Name"
format:
  html:
    toc: true
    toc-depth: 2
    embed-resources: true
---


```{r setup}
#| include: false
knitr::opts_chunk$set(
  collapse = TRUE, 
  warning = FALSE,
  message = FALSE,
  error = TRUE,
  fig.height = 2.75, 
  fig.width = 4.25,
  fig.env = 'figure',
  fig.pos = 'h',
  fig.align = 'center')
```


::: {.callout-note title = "Organize your files"}

This **qmd file** is where you'll type notes, code, etc.
Directions:

- Take notes in whatever way is best for you. You won't hand them in.
- Save this file in the "Activities" sub-folder of the "STAT 155" folder you created before today's class. Use a file name related to the activity number and/or today's date (eg: "activity 3" or "3 simple linear regression").
:::


\

::: {.callout-note title = "Learning goals"}

Explore numerical and visual summaries for the relationship between a *quantitative* **response (aka outcome) variable** and a *quantitative* **predictor (aka explanatory variable)**:

- Construct and interpret a **scatterplot visualization** of the relationship.
- Compute and interpret the linear **correlation** between the 2 variables.
- Analyze and apply a simple linear regression model of the relationship:   
    - obtain and write a **model formula**
    - interpret the model's **intercept** and **slope** coefficients *in context*
    - compute and interpret **predictions** (aka **expected** or **fitted values**) and **residuals** *in context*
    - explain the connection between residuals and the **least squares criterion**

![](https://ajohns24.github.io/155_Fall_2025/images/concept_map_3.png)

:::



\




::: {.callout-note title="Additional resources"}

**Required videos**

- [Simple linear regression Part 1: motivation & scatterplots](https://youtu.be/ghweQRQE0yw) ([slides](https://drive.google.com/file/d/18EzQF4oids5XXWZWDo5CABZYaOXhTbjZ/view?usp=sharing))
- [Simple linear regression Part 2: correlation](https://youtu.be/S6sgLrROfNw) ([slides](https://drive.google.com/file/d/18EzQF4oids5XXWZWDo5CABZYaOXhTbjZ/view?usp=sharing))
- [Simple linear regression Part 3: simple linear regression models](https://youtu.be/cwxUNJUYKsA) ([slides](https://drive.google.com/file/d/18EzQF4oids5XXWZWDo5CABZYaOXhTbjZ/view?usp=sharing))


**Optional**

- Reading: Sections 2.8, 3.1-3.3, 3.6 in the [STAT 155 Notes](https://mac-stat.github.io/Stat155Notes/)
- Videos:   
    - [R Code for Fitting a Linear Model](https://voicethread.com/share/14928147/) (Time: 11:07)

:::


\
\
\
\


# References {-}

**Correlation**

A numerical measure of the **strength** and **direction** of the **linear** relationship between two **quantitative** variables, Y and X.
Properties:

- $-1 \le r \le 1$
- the *sign* of $r$ indicates the *direction* of a linear relationship
- the *magnitude* of $r$ indicates the *strength* of a linear relationship. Consider the extremes:    
    - $r = 0$ indicates *no* linear relationship
    - $r = 1$ indicates a *perfect* positive linear relationship
    - $r = -1$ indicates a *perfect* negative linear relationship

::: {.callout-note title = "OPTIONAL MATH BOX: Correlation"}

The correlation $r$ of 2 quantitative variables $Y$ and $X$ is the (almost) **average** of products of their z-scores:

$$r = \frac{\sum z_x z_y}{n - 1} = \frac{\left(\frac{y_1 - \overline{y}}{s_y}\right)\left(\frac{x_1 - \overline{x}}{s_x}\right) + \left(\frac{y_2 - \overline{y}}{s_y}\right)\left(\frac{x_2 - \overline{x}}{s_x}\right) + \cdots + \left(\frac{y_n - \overline{y}}{s_y}\right)\left(\frac{x_n - \overline{x}}{s_x}\right)}{n - 1}$$
where $(y_1, y_2, ..., y_n)$ are the observed data points of $Y$ with mean $\overline{y}$ and standard deviation $s_y$ (similarly for the $X$ terms).

:::


\
\


**Population simple linear regression model**

Let 

- $Y$ = a quantitative **response variable**    

- $X$ = a quantitative **predictor** variable

Then the **(population) simple linear regression model** of the relationship of $Y$ with $X$ is:

$$E[Y | X] = \beta_0 + \beta_1 X$$

where

- $E[Y | X]$ is the expected or average value of $Y$ at a given $X$ value
- $\beta_0$ ("beta 0") is the y-intercept. It describes the typical value of $Y$ when $X = 0$.
- $\beta_1$ ("beta 1") is the slope. It describes the change in expected or average value of $Y$ associated with a 1-unit increase in $X$.

\
\


**Sample simple linear regression model**

In practice, we don't observe the entire population, thus don't know the population model.
Instead, we *estimate* this model using *sample* data.
The result is the **(sample) simple linear regression model**:

$$E[Y | X] = \hat{\beta}_0 + \hat{\beta}_1 X$$

where the $\hat{\beta}$ ("beta hat") terms are *estimates* of the actual $\beta$ values.



\
\


**Least squares estimation**    

We calculate the sample *estimates* $\hat{\beta}_0$ and $\hat{\beta}_1$ using the **least squares criterion**:
Choose the line that produces the smallest collection / sum of squared residuals, i.e. the "least squares".


```{r echo = FALSE}
# DON'T WORRY ABOUT THIS CODE
library(tidyverse)
set.seed(155)
data.frame(x = rnorm(50)) %>% 
  mutate(y = 2*x + 1 + rnorm(50, sd = 2)) %>% 
  mutate(predicted = lm(y ~ x, .)$fitted) %>% 
  ggplot(aes(x = x, y = y)) +
  geom_segment(aes(x = x, xend = x, y = y, yend = predicted), color = "red") +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE)
```


::: {.callout-note title = "OPTIONAL MATH BOX: Least squares estimates"}

Let 

- $(y_1, y_2, ..., y_n)$ = observed data
- $E[Y|X] = \hat{\beta}_0 + \hat{\beta}_1 X$ = a possible sample model
- $(\hat{y}_1, \hat{y}_2, ..., \hat{y}_n)$ = model predictions, e.g. $\hat{y}_1 = \hat{\beta}_0 + \hat{\beta}_1 x_1$
- $(y_1 - \hat{y}_1, y_2 - \hat{y}_2, ..., y_n - \hat{y}_n)$ = model residuals
- $\sum_{i=1}^n (y_i-\hat{y}_i)^2$ = sum of squared residuals

Then the $\hat{\beta}_0$ and $\hat{\beta}_1$ that *minimize* the sum of squared residuals are:

$$\begin{split}
\hat{\beta}_1 & = \frac{\sum_{i=1}^n(y_i - \overline{y})(x_i - \overline{x})}{\sum_{i=1}^n(x_i - \overline{x})^2} \\
\hat{\beta}_0 & = \overline{y} - \hat{\beta}_1 \overline{x} \\
\end{split}$$

:::




\
\
\
\





# Warm-up {.unnumbered}

**Guiding question**

How is bikeshare ridership related to various weather factors?


\


**Context**

We'll explore data from [Capital Bikeshare](https://capitalbikeshare.com/), a company in Washington DC.
Our main goal will be to explore daily ridership among *registered* users, as measured by `rides`.

```{r}
# Load packages and import data
# Shorten the "riders_registered" variable to "rides"
# Keep only certain columns / variables
library(tidyverse)
bikes <- read_csv("https://mac-stat.github.io/data/bikeshare.csv") %>% 
  rename(rides = riders_registered) %>% 
  select(date, day_of_week, weekend, temp_feel, humidity, windspeed, rides)
```


\
\


**EXAMPLE 1: Get to know the data**

```{r}
# Check out the first 6 rows
# What does a case represent, i.e. what's the unit of observation?


# How much data do we have?

```

**NOTE**

It's important to think about the "who, what, when, where, why, and how" of a dataset.
In this example, the *where* and *when* are particularly relevant.
These data were collected in D.C. in 2011-12.
The patterns we observe may not extend to other places and/or time periods!






\
\
\
\


**EXAMPLE 2: Explore ridership**

Let's get acquainted with the `rides` variable.
Remember: Code = communication.
Pay special attention to formatting / readability.

```{r}
# Calculate the average (central tendency) and standard deviation (variability) in daily ridership
bikes %>% 
  ___(mean(___), sd(___))
```

```{r}
# Construct a plot of how ridership varies from day to day

```


**Follow-up**

Summarize, in words, what you learned from the plot and numerical summaries.





\
\
\
\


**EXAMPLE 3: Relationships**

Now that we better understand ridership (`rides`), let's explore whether it's *related to* or might be *predicted by* various weather factors such as `windspeed` in miles per hour (mph), or the "feels like" temperature in degrees Fahrenheit (`temp_feel`).
In this setting:

- response variable Y = `rides`
- potential predictors X = `windspeed`, `temp_feel`

Let's start with the relationship of `rides` with `windspeed`.
We can visualize the relationship between these 2 quantitative variables using a **scatterplot**.
Suppose 4 friends have observed the scatterplot and describe it to you.
For each, discuss what the description is lacking!
What important info *doesn't* it tell you about the relationship?


a. "There's a weak association between ridership and wind speed." (Don't worry about the numbers on the x and y axis.)

b. "There's a weak, negative association between ridership and wind speed." (Don't worry about the numbers on the x and y axis.)

c. "There's a negative linear association between ridership and wind speed." (Don't worry about the numbers on the x and y axis.)



\
\
\
\


**EXAMPLE 4: Ridership vs temperature**

Next, let's explore the relationship of `rides` with `temp_feel`.

a. Let's build a scatterplot! This will require some new code, but let's start with what we know.

```{r}

```

b. Add the following two lines to your plot and describe what you observe.

```{r}
# Add a blue simple linear regression line (line of best fit)
# and a orange *curve* of best fit
PUT YOUR PLOT HERE +
  geom_smooth(method = "lm", se = FALSE) +
  geom_smooth(color = "orange", se = FALSE)
```




\
\
\
\


**EXAMPLE 5: Filtering our data**

Though we'll soon learn techniques for handling nonlinear relationships, right now let's focus on days with feels-like temperatures below 80 degrees (where the relationship is linear).
In the `tidyverse` package, whereas `select()` subsets our data to include only certain *columns / variables* of interest, `filter()` subsets our data to include only certain *rows / cases* of interest.
Let's practice:

```{r}
bikes %>% 
  filter(rides > 6911)
```

```{r}
bikes %>% 
  filter(rides >= 6911)
```

```{r}
bikes %>% 
  filter(rides == 6911)
```

Your turn: create a new dataset called `bikes_sub` that only keeps cases where `temp_feel` is under 80 degrees.

```{r}
# Pro tip: First convince yourself that your filter code is correct.
# THEN store the results as bikes_sub.
bikes %>% 
  filter(temp_feel ___)
```






\
\
\
\


**EXAMPLE 6: Correlation**

```{r}
bikes_sub %>% 
  ggplot(aes(y = rides, x = temp_feel)) + 
    geom_point() + 
    geom_smooth(method = "lm", se = FALSE)
```

Guess the correlation between ridership and temperature.
(You'll calculate this in the exercises below.)

\
\
\
\

**EXAMPLE 7: Simple linear regression (Part 1)**

```{r}
bikes_sub %>% 
  ggplot(aes(y = rides, x = temp_feel)) + 
    geom_point() + 
    geom_smooth(method = "lm", se = FALSE)
```

Consider the *sample* linear regression model of this relationship, i.e. the formula for the line in the scatterplot:

E[ridership | temperature] = $\hat{\beta}_0$ + $\hat{\beta}_1$ temperature

In the exercises, you'll show that:

E[ridership | temperature] = -2486.41 + 86.49 temperature

Consider the **intercept coefficient** -2486.41.
Physically, this is the y-intercept of our model line (where it crosses the y-axis).
What's the *contextual* meaning?

a. A 1 degree increase in temperature is associated with a 2486.41 decrease in average ridership.
b. On 0-degree days, we expect -2486.41 riders. (This is "correct" but doesn't make sense here since it **extrapolates** our model far beyond the range of our observed data.)


\
\
\
\



**EXAMPLE 8: Simple linear regression (Part 2)**

Consider the **temperature coefficient** 86.49.
Physically, this is the slope of our model line (rise over run).
Pick the most effective / correct *interpretation* of this slope below.
For each other option, indicate what's wrong with the interpretation.

a. Increasing X by 1 is associated with a 86.49 increase in Y.
b. A 1 degree increase in temperature is associated with a 86.49 increase in average ridership.
c. Warmer weather encourages ridership. A 1 degree increase in temperature produces 86.49 more riders on average.
d. If tomorrow is 1 degree warmer than today, we should expect 86.49 more riders.
e. A temperature change of 1 is associated with a 86.49 increase in average ridership.
 



\
\
\
\


::: {.callout-note title = "Interpreting model coefficients"}

When interpreting model coefficients, remember to:

1. interpret in context
2. use non-causal language
3. include units
4. talk about averages rather than individual cases

:::



\
\
\
\




# Exercises {-}

**DIRECTIONS**

You'll work on these exercises in your groups. 
Collaboration is a key learning goal in this course.

-   Why? Discussion & collaboration deepens your understanding of a topic while also improving confidence, communication, community, & more. (eg: Deeply learning a new language requires more than working through Duolingo alone. You need to talk with and listen to others!)

-   How? You are expected to:

    -   Use your group members' names & pronouns. It's ok to ask if you don’t remember!
    -   Actively contribute to discussion. Don't work on your own.
    -   Actively include all other group members in discussion.
    -   Create a space where others feel comfortable sharing ideas & questions.

-   We won't discuss these exercises as a class. With that, when you get stuck:    
    -   Carefully re-read the problem. Make sure you didn't miss any directions -- it can be tempting to skip words and go straight to an R chunk, but don't :).
    -   Discuss any questions you have with your group.
    -   If the question is unresolved by the group, be sure to ask the instructor!
    -   Remember that there are solutions in the online manual, at the bottom of the activity.



\
\
\
\


## Exercise 1: Correlation {-}

In the warm-up, you took a guess at the correlation between ridership and temperature:

```{r}
bikes_sub %>% 
  ggplot(aes(y = rides, x = temp_feel)) + 
  geom_point() + 
  geom_smooth(method = "lm", se = FALSE)
```

Let's check your intuition!
Fill in the code below to calculate the correlation.

```{r}
# HINT: What function is useful for calculating statistics (eg: means)?
bikes_sub %>% 
  ___(cor = cor(rides, temp_feel))
```



\
\
\
\


## Exercise 2: Build the model {-}

You were told above that our (sample) simple linear regression model was:

E[ridership | temperature] = -2486.41 + 86.49 temperature

Verify this using the code below.
Step through code chunk slowly, make note of new code.

```{r}
# Construct and save the model as bike_model_1
# What's the purpose of "rides ~ temp_feel"?
# What's the purpose of "data = bikes_sub"?
bike_model_1 <- lm(rides ~ temp_feel, data = bikes_sub)
```

```{r}
# A long summary of the model stored in bike_model_1
summary(bike_model_1)
```

```{r}
# A simplified model summary
coef(summary(bike_model_1))
```


\
\
\
\



## Exercise 3: Predictions and residuals {-}

On August 17, 2012, the `temp_feel` was 53.816 degrees and there were 5665 riders:
    
```{r}
# Note the use of filter() and select()!
bikes_sub %>% 
  filter(date == "2012-08-17") %>% 
  select(rides, temp_feel) 
```

### Part a {-}

In the scatterplot below, locate the point that corresponds to August 17, 2012 (by eye, not using code).
Is it close to the trend?
Were there *more* riders than expected or *fewer* than expected?

```{r}
bikes_sub %>% 
  ggplot(aes(y = rides, x = temp_feel)) + 
  geom_point() + 
  geom_smooth(method = "lm", se = FALSE)
```


### Part b {-}

Using the model formula below, calculate how many riders we would have *expected* or *predicted* on August 17, 2012 based on its temperature:

E[ridership | temperature] = -2486.41 + 86.49 temperature

```{r}
# Calculate the prediction
# (Convince yourself that you can connect this to the scatterplot)

```


### Part c {-}

Check your prediction using the `predict()` function.

```{r}
# What is the purpose of including bike_model_1?
# What is the purpose of newdata = ___???
predict(bike_model_1, newdata = data.frame(temp_feel = 53.816))
```


### Part d {-}

How far does the *observed* ridership on August 17, 2012 fall from its *model prediction*?
Calculate the **residual** or **prediction error**:

```{r}
# Recall: residual = observed Y - predicted Y

```


### Part e {-}

Are *positive* residuals above or below the trend line?

When we have *positive* residuals, does the model over- or under-estimate ridership?

Are *negative* residuals above or below the trend line?

When we have *negative* residuals, does the model over- or under-estimate ridership?





\
\
\
\


## Exercise 4: Ridership vs windspeed (visualization) {-}

Your turn!
Let's practice what you've learned in the preceding exercises to investigate the relationship between `rides` and `windspeed`.
Go back to using the `bikes` dataset (instead of `bikes_sub`) because we no longer need to only keep days less than 80 degrees!!!
    
### Part a {-}

```{r}
# Construct a visualization of this relationship
# Include a representation of the relationship trend
# Remember to use the bikes data!!!

```

### Part b {-}

Summarize, in words, your findings from the plot.



\
\
\
\

## Exercise 5: Ridership vs windspeed (model) {-}

### Part a {-}

```{r}
# Use lm() to construct a model of rides vs windspeed
# Save this as bike_model_2
# Remember to use the bikes data!!!

```

```{r}
# Get a short summary of bike_model_2

```

### Part b {-}

Write out a formula for the model trend.



### Part c {-}

Interpret the intercept *and* the windspeed coefficients.


### Part d {-}


```{r}
# Identify the windspeed on August 17, 2012 


# Predict ridership on this date using bike_model_2
# NOTE: Use predict() but also convince yourself you could do this from scratch


# Calculate the residual


```



\
\
\
\


## Exercise 6: Exploring correlation {-}

When analyzing data, it's important to: (1) understand the limitations and meaning of the tool we're applying; *and* (2) to visualize our data.
Calculating and using correlations are a perfect example.

For this exercise, we'll work with the `anscombe` dataset, which is built in to R:

```{r}
# Load anscombe data
data("anscombe")

# Check it out
head(anscombe)
```

Note that the `anscombe` dataset contains four different pairs of quantitative variables.
Below are the correlations between these pairs:

```{r}
# Correlation between x1, y1
anscombe %>% 
  summarize(cor(x1, y1))

# Correlation between x2, y2
anscombe %>% 
  summarize(cor(x2, y2))

# Correlation between x3, y3
anscombe %>% 
  summarize(cor(x3, y3))

# Correlation between x4, y4
anscombe %>% 
  summarize(cor(x4, y4))

```

### Part a {-}

What do you notice about each of these correlations?


### Part b {-}

Discuss what you **think** the scatterplots for these pairs of variables might look like.


### Part c {-}

Check out separate scatterplots for each of the 4 pairs of variables.
Is this what you expected?
What is the *message* of this last exercise as it relates to the limits of correlation?

```{r}
# Ignore this code!
# It wrangles the data to nicely put all scatterplots on the same frame
anscombe %>%
  pivot_longer(cols = everything(),
               names_to = c(".value", "group"),
               names_pattern = "(.)(\\d)") %>%
  ggplot(aes(y = y, x = x)) + 
  geom_smooth(method = "lm", se = FALSE) +
  geom_point() + 
  facet_wrap(~ group)
```






\
\
\
\


## Exercise 7: Data drill (`filter`, `select`, `summarize`) {-}

This exercise is designed to help you practice your tidy data wrangling skills.
We can't do statistics or really understand data without these skills!
We'll work with a simpler set of 10 data points:

```{r}
new_bikes <- bikes %>% 
  select(date, temp_feel, humidity, rides, day_of_week) %>% 
  head(10)
```

### Part a {-}

Thus far, in the `tidyverse` grammar you've learned 3 **verbs** or action words: `summarize()`, `select()`, `filter()`.
First, refresh your memory.
Take note of the following code and the output it produces:
  
```{r}
# First check out the data for a point of reference
new_bikes
```

```{r}
# summarize()
new_bikes %>% 
  summarize(mean(temp_feel), mean(humidity))
```
    
```{r}
# select()
new_bikes %>%
  select(date, temp_feel)
```

```{r}
# select() again
new_bikes %>% 
  select(-date, -temp_feel)
```

```{r}
# filter()
new_bikes %>% 
  filter(rides > 850)
```

```{r}
# filter() again
new_bikes %>% 
  filter(day_of_week == "Sat")
```

```{r}
# filter() again
new_bikes %>% 
  filter(rides > 850, day_of_week == "Sat")
```


### Part b {-}

Your turn.
Use `tidyverse` verbs to complete each task below.
   
```{r}
# Keep only information about the humidity and day of week

# Keep only information about the humidity and day of week using a different approach

# Calculate the maximum and minimum temperatures

# Keep only information for Sundays

# Keep only information for Sundays with temperatures below 50

# Calculate the median ridership on Sundays with temperatures below 50

```



\
\
\
\




# Reflection {-}


**Learning goals**

Go to the top of this file and review the learning goals for today's activity.

- Which do you have a good handle on?

- Which are struggling with? What feels challenging right now?

- What are some wins from the day?

- Statistics is a particular kind of language and collection of tools for channeling **curiosity** to improve our world. How do the concepts we practiced today facilitate curiosity?


**Code**

In addition to exploring the learning goals, you learned some new code.

- If you haven't already, you're highly encouraged to start tracking and organizing new code in a cheat sheet (eg: a Google doc). This will be a handy reference for you, and the act of making it will help deepen your understanding and retention.

- Reflect upon the `lm()` function. This function takes 2 arguments. What are they and what's their purpose? Mainly, what information does R need in order to build a model?


- Reflect upon the `predict()` function. This function takes 2 arguments. What are they and what's their purpose? Mainly, what information does R need in order to make a prediction?




## Done!

- Finalize your notes: (1) Render your notes to an HTML file; (2) Inspect this HTML in your Viewer -- check that your work translated correctly; and (3) Outside RStudio, navigate to your 'Activities' subfolder within your 'STAT155' folder and locate the HTML file. Open and inspect this in a web browser.
- Clean up your RStudio session: End the rendering process by clicking the 'Stop' button in the 'Background Jobs' tab in the lower left pane.
- Check the wrap-up announcements and solutions in the online course manual, at the bottom of the corresponding chapter.
- Work on homework!


