Logo
PREPITUS
Data AnalysisAdvanced

Evaluating Statistical Claims: Observational Studies and Experiments

Match every study design to the strongest conclusion it can support — and spot when a claim outruns its evidence.

⏱ 7 min📍 Learn first: Inference from Sample Statistics
Start Free LessonTake 4-Question Check
Free KC Learning Toolkit

Learn Evaluating Statistical Claims: Observational Studies and Experiments in the format that works for you

Start with a quick audio review, take the deeper podcast lesson, or move directly into the visual guide and knowledge check below.

13 free resources
00:00 / 02:07

How to spot misleading statistical claims

A short spoken refresher on this concept.

00:00 / 22:20

How Statistical Averages Hide the Truth

The full lesson, worked through in depth.

Concept Snapshot

Definition
Observational studies watch existing groups; experiments randomly ASSIGN treatments. Only random assignment supports cause-and-effect.
What the SAT tests
Choosing the appropriate conclusion given how subjects were selected and whether treatments were assigned randomly.
Key rule
Random SELECTION → generalize to the population. Random ASSIGNMENT → conclude causation. Each independently; both, either, or neither.
Most common mistake
Concluding causation from an observational study because the sample was large or random.
Prerequisite
Inference from Sample Statistics

Visual Explanation

What to picture: Random SELECTION → generalize to the population. Random ASSIGNMENT → conclude causation. Each independently; both, either, or neither.

A diagram for this concept is in production. Until it is ready, the worked example below shows the same relationship step by step.

Concept Mind Map

DefinitionObservational studies watch existing groups; experiments randomly ASSIGN treatments. Only random assignment supports cause-and-effect.
Key ruleRandom SELECTION → generalize to the population. Random ASSIGNMENT → conclude causation. Each independently; both, either, or neither.

Evaluating Statistical Claims: Observational Studies and Experiments

Data Analysis

Best ConclusionGiven the design, which conclusion is appropriate?
Design FlawWhat limits this study — selection bias, no control group, confounding?
Two RandomsTrack selection and assignment separately.

How the SAT Tests It

Pattern

Best Conclusion

Given the design, which conclusion is appropriate?

Clue: "which conclusion is best supported"
Pattern

Design Flaw

What limits this study — selection bias, no control group, confounding?

Clue: "volunteers", "chose to participate"
Pattern

Two Randoms

Track selection and assignment separately.

Clue: "randomly selected" vs. "randomly assigned"

Worked Example

Researchers randomly select 500 adults, then observe that those who drink green tea have lower blood pressure. What can they conclude?

Check the two randoms separately: selection ✓ random, assignment ✗ none — subjects chose their own tea habits.
Random selection → the ASSOCIATION generalizes to the adult population.
No random assignment → no causal claim. Tea drinkers may differ in diet, exercise, income — confounders roam free.
Green tea drinking is associated with lower blood pressure among adults — nothing about tea CAUSING it.

Try It Yourself

A sleep scientist recruits 200 volunteers, randomly assigns half to a screens-off-by-10pm rule and half to no rule, and finds the screens-off group sleeps 48 minutes longer. Which conclusion is best supported?

Common Traps

What students do

Letting one random do both jobs.

Why it fails

Random selection and random assignment license different conclusions — a study can have either, both, or neither.

Do this instead

Draw two checkboxes on every study question: SELECTED randomly? ASSIGNED randomly? Match the conclusion to the checked boxes only.

Interactive Flashcards

Review deckCard 1 of 8

Four-Question Knowledge Check

4 quick questions drawn from this concept's own snapshot, patterns and traps.

Question 1 of 4. Which statement is the key rule for this concept?

Audio Lesson

How Statistical Averages Hide the Truth

The selection/assignment framework for study questions.

00:00 / 22:20

22:20 · the full lesson

What to notice

Random SELECTION → generalize to the population. Random ASSIGNMENT → conclude causation. Each independently; both, either, or neither.

What the lesson covers

  • 0:00 Observation vs experiment
  • 1:30 The two checkboxes
  • 3:00 Confounders
  • 4:20 Practice

Study Guide and Transcript

One-page study guide

The snapshot, the key rule, the patterns and the traps, on one page.

Source-Grounded AI Tutor

Ask the Evaluating Statistical Claims: Observational Studies and Experiments notebook

A tutor grounded in this concept's own sources — ask for a simpler explanation, another example, or why a particular answer is wrong. It is in development and will appear here once its sources pass academic review.

Ready to lock this one in?

Practise Evaluating Statistical Claims: Observational Studies and Experiments against real Digital SAT questions, with your results tracked.

Practise this concept

or continue to Area and Volume

Try a questionPractise this concept