Class Schedule¹

We recommend reading the papers in the order they are listed.

Week 01 Section: Thursday, 09/03 The scientific method as it pertains to psychology [Gary]

(We’ll use up the whole 9:30-10:45am timeslot this week)

• Readings (in class): Forscher (1963); Chiang (2000)
• Readings for after class: Feynman (1974); Gelman & Higgs (2025);

Week 02, Tuesday, 09/08 Experiments, causality, and the two disciplines of psychology [Gary]

• Readings: Cohen (1994); Experimentology: Experiments & causal inference; Experimentology: How data inform theoretical constructs; Cronbach (1957)

Section: Thursday, 09/10

• Read Smaldino (2017); discussion of when should we (not) do experiments? A discussion of what (non)experimental methods can and can’t tell us. What are good uses of modeling? Also read Experimentology: Primer on research ethics. [Joao]

Week 03, Tuesday, 09/15 Measurement: Validity, reliability, the reliability paradox [Joao]

• Readings: Experimentology: Measurement; Westfall & Yarkoni (2016); Hedge et al. (2018); Zorowitz & Niv (2023)

• Extra: Sijtsma (2009); Cortina et al. (2020);

Section: Thursday, 09/17

• With a partner, prepare a short (~5 minute) presentation on what you know and don’t know about the reliability and validity of the measures you are using in your research and how you could go about further improving them or (2) Reflect on the money–>happiness example and come up with a validation strategy for a specific measure of happiness. In either case, discuss different types of validity. What type of data collection would this validation effort require? [Gary]

Week 04, Tuesday, 09/22 Experimental Design 1: Design basics; the ergodic fallacy [Gary]

• Readings: Experimentology: Design; Speelman & McGann (2020)

• Extra: Speelman et al. (2024); Rouder & Haaf (2019)

Section: Thursday, 09/24

• [Gary]

Week 05, Tuesday, 09/29 Sampling, power, and effect sizes [Gary]

• Readings: Experimentology: Sampling; Funder & Ozer (2019); Smallest Effect Size of Interest

• Extra: Wagenmakers (2007); Lovakov & Agadullina (2021); Marek et al. (2022)

Section: Thursday, 10/01

• [Joao]

Week 06, Tuesday, 10/06 Data management best practices [Joao]

• Readings: Experimentology: Project management; Wilkinson et al. (2016)

Section: Thursday, 10/08

• [Joao]

Week 07, Tuesday, 10/13 Experimental Design 2: Best practices in running behavioral studies; data quality checks [Gary]

• Readings: Experimentology: Data collection; Barbosa et al. (2023); Westwood (2025);

• Extra: Kay (2025); Anders et al. (2026)

Section: Thursday, 10/15

• Practice with designing effective data quality checks [Joao]

Week 08, Tuesday, 10/20 Experimental Design 3: Closer look at some common/useful behavioral methods [Gary]

• Readings: Skim Bower & Clapper, pausing on what most interests you & reading those parts more thoroughly Bower & Clapper (1989); skim over Leek (2001) to make sure you understand the goal and key parts of these adaptive methods; Look over the first chapter or two of Macmillan & Creelman (2005)

Section: Thursday, 10/22

• [Gary]

Week 09, Tuesday, 10/27 Quantifying Qualitative Data [Joao]

• Readings: Charlesworth et al. (2024); Lin et al. (2023)

Section: Thursday, 10/29

• [Joao]

Week 10, Tuesday, 11/03 Non-Experimental Design 1: Surveys; Item Response Theory; Propensity score matching [Joao]

• Readings: Reise & Waller (2009); Clark & Watson (2016)

Section: Thursday, 11/05

• [Joao]

Week 11, Tuesday, 11/10 Non-Experimental Methods 2: Longitudinal & In-the-Wild Data [Joao]

• Readings: Goldstone & Lupyan (2016)

Section: Thursday, 11/12

• [Gary]

Week 12, Tuesday, 11/17 Meta-Analysis [Gary]

• Readings: Experimentology: Meta-Analysis

Section: Thursday, 11/19

• [Joao] - With a partner, prepare a short (~5 minute) presentation on a meta-analysis paper of your choice (we will provide some example papers). What was the outstanding question? What methods did the authors use in meta-analyzing the literature? What did the meta-analysis find and how does it match up with claims made in the individual papers? Did it reveal any publication bias? In your judgment, do we now have a more ‘final’ word on whether the effect in question is real?

Week 13, Tuesday, 11/24 No Class (Thanksgiving travel)

Section: Thursday, 11/26

• Happy Thanksgiving!

Week 14, Tuesday, 12/01 Putting it all together: Maximizing knowledge gain [Gary]

• Readings: Yarkoni (2020); Giner-Sorolla (2019); Rozin (2009)

Section: Thursday, 12/03

• Peer review of grant proposals [Gary & Joao]

Week 15, Tuesday, 12/08 Presentations of your grant proposals [Gary & Joao]