maths.free › Statistics & Probability › 8. Statistics › Scatter Plots, Correlation, and Regression Lines

Scatter Plots, Correlation, and Regression Lines

Construct a scatter plot for a dataset.

Learning Objectives

After completing this section, you should be able to:

  1. Construct a scatter plot for a dataset.
  2. Interpret a scatter plot.
  3. Distinguish among positive, negative and no correlation.
  4. Compute the correlation coefficient.
  5. Estimate and interpret regression lines.

Relationships Between Quantitative Datasets

Before we can evaluate a relationship between two datasets, we must first decide if we feel that one might depend on the other. In our exam example, it is appropriate to say that the score on the final depends on the score on the midterm, rather than the other way around: if the midterm depended on the final, then we’d need to know the final score first, which doesn’t make sense.

Here’s another example: if we collected data on home purchases in a certain area, and noted both the sale price of the house and the annual household income of the purchaser, we might expect a relationship between those two. Which depends on the other? In this case, sale price depends on income: people who have a higher income can afford a more expensive house. If it were the other way around, people could buy a new, more expensive house and then expect a raise! (This is very bad advice.)

It's worth noting that not every pair of related datasets has clear dependence. For example, consider the percent of a country’s budget devoted to the military and the percent earmarked for public health. These datasets are generally related: as one goes up, the other goes down. However, in this case, there’s not a preferred choice for dependence, as each could be seen as depending on the other. When exploring the relationship between two datasets, if one set seems to depend on the other, we’ll say that dataset contains values of the response variable (or dependent variable). The dataset that the response variable depends on contains values of what we call the explanatory variable (or independent variable). If no dependence relationship can be identified, then we can assign either dataset to either role.

Once we’ve assigned roles to our two datasets, we can take the first step in visualizing the relationship between them: creating a scatter plot.

Condensed — the full section is in OpenStax Contemporary Mathematics.

Linear Regression

The final step in our analysis of the relationship between two datasets is to find and use the equation of the regression line. For a given set of explanatory and response data, the regression line (also called the least-squares line or line of best fit) is the line that does the best job of approximating the data.

What does it mean to say that a particular line does the “best job” of approximating the data? The way that statisticians characterize this “best line” is rather technical, but we’ll include it for the sake of satisfying your curiosity (and backing up the claim of "best"). Imagine drawing a line that looks like it does a pretty good job of approximating the data. Most of the points in the scatter plot will probably not fall exactly on the line; the distance above or below the line a given point falls is called that point’s residual. We could compute the residuals for every point in the scatter plot. If you take all those residuals and square them, then add the results together, you get a statistic called the sum of squared errors for the line (the name tells you what it is: “sum” because we’re adding, “squared” because we’re squaring, and “errors” is another word for “residuals”). The line that we choose to be the “best” is the one that has the smallest possible sum of squared errors. The implied minimization (“smallest”) is where the “least” in “least squares” comes from; the “squares” comes from the fact that we’re minimizing the sum of squared errors. This is very similar to the process we outlined in the "game" that we used to introduce the mean. Both the regression line and the mean are designed to minimize a sum of squared errors. Here ends the super technical part.

Condensed — the full section is in OpenStax Contemporary Mathematics.

Extrapolation

A very common misuse of regression techniques involves extrapolation, which involves making a prediction about something that doesn't belong in the dataset.

Condensed — the full section is in OpenStax Contemporary Mathematics.

Correlation Does Not Imply Causation

One of the most common fallacies about statistics has to do with the relationship between two datasets. In the dataset “Public”, we find that the correlation coefficient between the 75th percentile math SAT score and the 75th percentile verbal SAT score is 0.92, which is really strong. The slope of the regression line that predicts the verbal score from the math score is 0.729, which we might interpret as follows: “If the 75th percentile math SAT score goes up by 10 points, we’d expect the corresponding verbal SAT score to go up by just over 7 points.”

Does the increasing math score cause the increase in the verbal score? Probably not. What’s really going on is that there’s a third variable that’s affecting them both: To raise the SAT math score by 10 points, a school will recruit students who do better on the SAT in general; these students will also naturally have higher SAT verbal scores. This third variable is sometimes called a lurking variable or a confounding variable. Unless all possible lurking variables are ruled out, we cannot conclude that one thing causes another.

Condensed — the full section is in OpenStax Contemporary Mathematics.

Key Concepts

  • If one variable affects the value of another variable, we say the first is an explanatory variable and the second is a response variable.
  • Scatter plots place a point in the \(\text{xy}\)-plane for each unit in the dataset. The \(x\)-value is the value of the explanatory variable, and the \(y\)-value is the value of the response variable.
  • The correlation coefficient \(r\) gives us information about the strength and direction of the relationship between two variables. If \(r\) is positive, the relationship is positive: an increase in the value of the explanatory variable tends to correspond to an increase in the value of the response variable. If \(r\) is negative, the relationship is negative: an increase in the value of the explanatory variable tends to correspond to a decrease in the value of the response variable. Values of \(r\) that are close to 0 indicate weak relationships, while values close to –1 or indicate strong relationships.
  • The regression line for a relationship between two variables is the line that best represents the data. It can be used to predict values of the response variable for a given value of the explanatory variable.

Formulas

If a line has slope \(m\) and passes through a point \(({x}_{0},{y}_{0})\), then the point-slope form of the equation of the line is: \[y=m(x-{x}_{0})+{y}_{0}\]

Suppose \(x\) and \(y\) are explanatory and response datasets that have a linear relationship. If their means are \(\overset{\bar}{x}\) and \(\overset{\bar}{y}\) respectively, their standard deviations are \({s}_{x}\) and \({s}_{y}\) respectively, and their correlation coefficient is \(r\), then the equation of the regression line is: \[y=r(\frac{{s}_{y}}{{s}_{x}})(x-\overset{\bar}{x})+\overset{\bar}{y}.\]

Projects

  1. Browse through some news websites to find five stories that report on data and include data visualizations. Can you tell from the report how the data were collected? Was randomization used? Are the visualizations appropriate for the data? Are the visualizations presented in a way that might bias the reader?
  2. We discussed three measures of centrality in this chapter: the mode, the median, and the mean. In a broader context, the mean as we discussed it is more properly called the arithmetic mean, to distinguish it from other types of means. Examples of these include the geometric mean, harmonic mean, truncated mean, and weighted mean. How are these computed? How do they compare to the arithmetic mean? In what situations would each of these be preferred to the arithmetic mean?
  3. Simpson’s Paradox is a statistical phenomenon that can sometimes appear when we observe a relationship within several subgroups of a population, but when the data for all thegroups are analyzed all together, the opposite relationship appears. Find some examples of Simpson’s Paradox in real-world situations, and write a paragraph or two that would explain the concept to someone who had never studied statistics before.

Practice (9)

Try each one on paper first. Reveal the answer to check; verified ones can be opened in the solver for every step.

  1. For each of the following pairs of related datasets, identify which (if any) should be assigned the role of response variable and which should be assigned to be the explanatory variable.

    1. A person’s height and weight
    2. A professional basketball player’s salary and their average points scored per game (which is a measure of how good they are at basketball)
    3. The length and width of leaves on a tree
    Otkrij odgovor
    1. As people get taller, their weight tends to increase. But if a person goes on a diet and loses weight, we don’t expect them to also get shorter. So, weight depends on height. That means we’ll say that the response variable is weight and the explanatory variable is height.
    2. The more points a basketball player scores, the more money they should make. But if a basketball player gets a raise, we wouldn’t expect them to get better at basketball as a result. So, the response is salary and the explanatory is points per game.
    3. The way that the length and width of leaves are connected isn’t clear. It seems reasonable that as the width goes up, so would the length. But the other direction is also plausible: as the length goes up, so does the width. Without a clear dependence relationship, we’re free to declare either to be the response and the other to be the explanatory.

  2. Students are exploring the relationship between scores on the midterm exam and final exam in their math course. Here are some of the scores reported by their classmates:

    NameMidterm gradeFinal grade
    Student 18884
    Student 27180
    Student 37577
    Student 49495
    Student 56873

    Create a scatter plot to visualize the data.

    Otkrij odgovor

    Step 1: Since it makes more sense to think of the final exam score as being dependent on the midterm exam score, we’ll let the final grade be the response. So, let’s think of these two datasets as a set of ordered pairs, midterm first, final second:

    Name(Midterm, Final)
    Student 1(88, 84)
    Student 2(71, 80)
    Student 3(75, 77)
    Student 4(94, 95)
    Student 5(68, 73)

    Step 2: Next, let’s make the axes. On the horizontal axis, make sure the range of values is sufficient to cover all of the explanatory data. For our data, that’s 68 to 94. Similarly, the vertical axis should cover all of the response data (73 to 95):

    Step 3: Our first point is (88, 84). So, we’ll locate 88 on the horizontal axis, 84 on the vertical axis, and identify the point that’s directly above the first location and horizontally level with the second:

    Step 4: Repeat this process to place the other four points on the graph:

    Step 5: Finally, label the axes:

  3. The dataset “NHL19” gives the results of the 2018–2019 National Hockey League season. The columns are team, wins (W), losses (L), overtime losses (OTL), total points (PTS), goals scored by the team (GF), goals scored against the team (GA), and goal differential (the difference in GF and GA). Use Google Sheets to create a scatter plot for GF vs. GA.

    Otkrij odgovor

    Step 1: Open the dataset in Google Sheets, and click and drag to select the data we want to visualize (in this case, we want the columns for GF and GA; make sure you include those labels in the selection).

    Step 2: Next, click on the “Insert” menu, then click “Chart.” Sheets will automatically choose a chart format; if the result isn’t a scatter plot, click on the drop-down menu under “Chart type” in the Chart Editor on the right side of the window and select “Scatter chart.”

    Step 3: Next, check that the correct choices were made for the horizontal and vertical axes. In this case, we want to see GF on the vertical axis and GA on the horizontal axis. If Sheets made the wrong choice, we can fix it in the Chart Editor by clicking on the name of the dataset under “X-axis” to open up a dropdown menu, then selecting the variable that should go on the horizontal axis (GA in this case).

    Step 4: Now, click on the variable under “Series” and select the one that should go on the vertical axis (GF). If you had to make that change, the axis labels in the graph may also need changing; those labels can be fixed using the “Customize” tab in the Chart Editor under “Chart & axis titles.” Your result should look like this:

  4. Consider the four scatter plots below:

    For each of these, answer the following questions:

    1. Is there a curved pattern in the data? If yes, stop here. If no, continue to part b.
    2. Classify the strength and direction of the relationship. Make a guess at the value of \(r\).
    Otkrij odgovor
      1. Yes, there is a curved pattern.
      1. No, there’s no curved pattern.
      2. Since the points tend upward as we move from left to right, this is a positive relationship. The points seem pretty closely grouped around a line, so it’s fairly strong. Comparing this scatter plot to those in to , we can see that the relationship is stronger than the one in (\(r=0.61\) ) but not as strong as the one in (\(r=0.97\)). So, the value of the correlation coefficient is somewhere between the two. We might guess that \(r=0.9\).
      1. No, there’s no curved pattern.
      2. Since the points tend downward as we move from left to right, this is a negative relationship. The points are not tightly grouped around a line, but the pattern is clear. It looks like it has approximately the same strength as the plot in , just with the opposite sign. So, we might guess that \(r=-0.6\).
      1. No, there’s no curved pattern.
      2. Since the points don’t really tend upward or downward as we move from left to right, there is no real relationship here. Thus, \(r\approx 0\).

  5. The data that were plotted in the previous example can be found in the dataset “correlationcoefficient1”. All of them share the same values for the explanatory variable \(x\). The four responses are labeled \(y1\) through \(y4\). Compute the correlation coefficients for each, if appropriate, using Google Sheets. Round to the nearest hundredth.

    Otkrij odgovor

    Step 1: There is a curved pattern in the data, so the correlation coefficient isn’t meaningful.

    Step 2: Using “=CORREL(A2:A101, C2:C101)” we get \(r=0.89\).

    Step 3: Using “=CORREL(A2:A101, D2:D101)” we get \(r=-0.66\).

    Step 4: Using “=CORREL(A2:A101, E2:E101)” we get \(r=-0.04\).

  6. Suppose you have datasets \(x\) and \(y\) with the following statistics: \(x\) has mean 21 and standard deviation 4, \(y\) has mean 8 and standard deviation 2, and their correlation coefficient is −0.4. What’s the equation of the regression line?

    Otkrij odgovor

    Step 1: We’re given \(\overset{\bar}{x}=21\), \({s}_{x}=4\), \(\overset{\bar}{y}=8\), \({s}_{y}=2\), and \(r=-0.4\). Let's start with the formula for the equation of the regression line: \[y=r(\frac{{s}_{y}}{{s}_{x}})(x-\overset{\bar}{x})+\overset{\bar}{y}\]

    Step 2: Plugging in our values gives us: \[y=-0.4(\frac{2}{4})(x-21)+8\]

    Step 3: Our final regression line equation is: \[y=-0.2x+12.2\]

  7. In , we considered the relationship between goals scored (GF) and goals against (GA) using the dataset “NHL19”. Recreate the scatter plot in Google Sheets, and use it to find the equation of the regression line.

    Otkrij odgovor

    Once we have recreated the scatter plot, we find the equation of the regression line by clicking the three dots at the top right of the plot, selecting “Edit chart,” then clicking on “Customize” and “Series.” We add the regression line by checking the box next to “Trendline,” and then we show the equation by selecting “Use Equation” in the drop-down menu under “Label.” The equation of the tangent line is \(y=-0.0554x+261\).

  8. The data in “MLB2019Off” gives offensive team stats for the 2019 Major League Baseball season. Use that dataset to answer the following questions:

    1. What is the equation of the regression line for runs (R) vs. hits (H)?
    2. How many runs would we expect a team to score if the team got 1500 hits in a season?
    3. Did the Kansas City Royals (KCR) overperform or underperform in terms of runs scored, based on their hit total? By how much?
    4. Write a sentence to interpret the slope of the regression line.
    Otkrij odgovor
    1. Using Google Sheets, we find that the regression line equation is \(y=0.884x-456\), where \(y\) is the number of runs scored and \(x\) is the number of hits.
    2. Plugging 1500 into the equation of the regression line, we get \(0.884\times 1500-456=870\). We would predict that a team with 1500 hits would score 870 runs.
    3. The Royals had 1356 hits, so we would predict their run total to be \(0.884\times 1356-456\approx 743\). They actually scored 691 runs, so they underperformed expectations by 52 runs \((743-691)\).
    4. The slope gives us the predicted change in the response that corresponds to a one unit increase in the explanatory variable. So, we expect one additional hit to result in 0.884 more runs. Since 0.884 runs doesn’t really make sense, we can get a better interpretation by multiplying through by ten or one hundred: Ten additional hits will result in almost nine additional runs, or a hundred additional hits will yield on average just over 88 additional runs.

  9. The data in “WNBA2019” gives team statistics from the 2019 WNBA season. Use that dataset to answer these questions about team wins (W) and the proportion of team field goals made (FG%, the number of shots made divided by the number of shots attempted. Even though this column is labeled using a percent sign, the values are not expressed as percentages):

    1. What is the equation of the regression line for wins vs. proportion of made field goals?
    2. How many wins would we expect for a team that makes 42% of its shots?
    3. Did the New York Liberty overperform or underperform in terms of wins, based on the team’s proportion of made field goals?
    4. Write a sentence to interpret the slope of the regression line.
    Otkrij odgovor
    1. Using Google Sheets, we get the equation \(y=178.097x-58.543\), where \(x\) is the proportion of field goals made and \(y\) is the number of wins.
    2. Since 42% corresponds to a proportion of 0.42, we’ll plug 0.42 into the regression equation for \(x\), which gives us \(178.097\times 0.42-58.543\approx 16.26\). We would predict that a team that makes 42% of its shots would win about 16 games.
    3. The New York Liberty made 41.4% of their shots, so we expect they would have \(178.097\times 0.414-58.543\approx 15.19\)wins. In fact, they had only 10 wins, so they underperformed expectations by over 5 wins.
    4. Step 1: The slope gives us the expected increase in the response that corresponds to a one unit increase in the explanatory variable. If we simply go with that interpretation, we would get a sentence like “We expect an increase in proportion of field goals made of 1 would result in an additional 178 wins.” However, that sentence doesn't make much sense. Let's consider why.
      Step 2: First, proportions must be between 0 and 1, the proportion of made field goals can’t be increased by 1 and still make sense. Second, the total number of games played is only 34, so no team could get an additional 178 wins! So, we’ll have to change the units.
      Step 3: Since the proportions of made field goals are often expressed as a percentage, we could try to use that. If we express the slope as a fraction with 1 in the denominator (remember, the denominator represents the proportion of field goals made), then convert the denominator to a percentage and simplify, we get \(\frac{178.097}{1}=\frac{178.097}{100\%}\approx \frac{1.78}{1\%}\).
      Step 4: So, an increase in field goal percentage of 1% would result in an expected increase of 1.78 wins.

Symbols used here

\pm
plus or minus
Both signs at once: x = 3 ± 2 means 5 and 1.
\approx
approximately equal
Equal to the precision shown, not exactly.
n!
factorial
n × (n−1) × … × 1; the number of orderings of n things. 0! = 1.
\binom{n}{k}
binomial coefficient, "n choose k"
Number of k-element subsets of n things: n!/(k!(n−k)!).
\sum_{k=1}^{n} a_k
summation
Add a_k for k = 1 up to n.
A \cup B,\ A \cap B,\ A \setminus B
union, intersection, difference
In either; in both; in A but not B.
\bar{x},\ \mu
sample mean, population mean
Average of the data; average of the whole population.
\sigma,\ s,\ \sigma^2
standard deviation, sample s.d., variance
Typical distance from the mean; its square.
P(A),\ P(A \mid B)
probability, conditional probability
Chance of A; chance of A given that B happened.
E[X],\ \operatorname{Var}(X)
expected value, variance
Probability-weighted average of X; its spread.
N(\mu, \sigma^2),\ z
normal distribution, z-score
The bell curve with mean μ and variance σ²; (x − μ)/σ.

How to: Scatter Plots, Correlation, and Regression Lines

  1. Construct a scatter plot for a dataset.
  2. Interpret a scatter plot.
  3. Distinguish among positive, negative and no correlation.
  4. Compute the correlation coefficient.
  5. Estimate and interpret regression lines.
  6. A person’s height and weight
  7. A professional basketball player’s salary and their average points scored per game (which is a measure of how good they are at basketball)
  8. The length and width of leaves on a tree

Questions people ask

Mean or median — which should I use?

Median when the data have outliers or a long tail (incomes, house prices); mean when the data are roughly symmetric and you want every value to count. Report both if they disagree — the gap is itself information.

What does a p-value actually say?

The probability of seeing data at least this extreme if the null hypothesis were true. It is not the probability that the null hypothesis is true.

Why divide by n − 1 for the sample variance?

The sample mean sits closer to the sample than the true mean does, so squared deviations from it are slightly too small on average; dividing by n − 1 instead of n corrects the bias.

Pokušaj i ti

Parts of this page are adapted from OpenStax Contemporary Mathematics (CC BY-NC-SA 4.0). Condensed and re-explained here; errors are ours.

Više u Statistics & Probability

Sampling and dataDescribing data with graphsMean, median and modeProbabilityCounting: permutations and combinationsDiscrete random variablesContinuous random variablesThe normal distributionThe central limit theoremConfidence intervalsHypothesis testingComparing two samplesChi-square testsLinear regression and correlation