Psychology 12: Capstone — a preregistered replication

Proof required

Link the time-stamped preregistration and the public repo with your materials (experiment code or the open dataset's link) and the consent text, if you collect data.

What gets checked

• The preregistration is dated before the data.
• The sample size has a power analysis.
• Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

• Preregistering after looking at the data.
• A plan too vague to check.

Powstik Guide

A dated preregistration and a public repo with the materials.

Steps

1. Pick the finding and read the original paper.
2. Choose open data or an anonymous adult survey.
3. Write the plan and power analysis.
4. Preregister it, then start.

Preregistration
Finding and original paper:
Hypothesis:
Design:
Sample size and power analysis:
Exact analysis:
Data rules (exclusions):

What gets sent back

• No date before the data.
• Data from minors or with health questions.

Resources

What a verifier looks for

• Is the preregistration dated before the data?
• Could someone else run the analysis from it?
• Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Proof required

Link the public repo with the anonymous data, the analysis code, the output, and a note listing every change from the plan.

What gets checked

• The planned analysis is run first and reported in full.
• Every deviation from the plan is listed.
• Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

• Dropping the planned test because it failed.
• Exploratory results presented as planned.

Powstik Guide

Anonymous data, analysis code and output in a public repo.

Steps

1. Collect or download the data.
2. Run the preregistered analysis.
3. List every change from the plan.
4. Label anything extra as exploratory.

Results
Planned test and result:
Effect size and interval:
Deviations from the plan:
Exploratory (labelled):

What gets sent back

• Analysis that differs from the plan without saying so.
• Identifiable data.

Resources

What a verifier looks for

• Does the code reproduce the reported numbers?
• Are deviations listed?
• Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Proof required

Link the public write-up with the repo and the recording of the defence, and name the people who questioned you.

What gets checked

• The write-up says plainly whether the finding replicated.
• Code, data and preregistration are linked from it.
• The defence answers questions about the method.

Common mistakes

• Spinning a failed replication as a success.
• A write-up with no data or code.

Powstik Guide

A public write-up with code, data and preregistration linked, and a recorded 15-minute defence.

Steps

1. Write it up in the standard order.
2. Link the preregistration, code and data.
3. Invite three people who know psychology.
4. Record the defence and its questions.

Write-up
Title:
Did it replicate (yes, no, partly):
Method:
Results:
What this means and what it doesn't:

What gets sent back

• No preregistration linked.
• No defence recording.

Resources

What a verifier looks for

• Does the conclusion follow from the planned test?
• Is everything needed to repeat it public?
• Were hard questions answered honestly?

You'll sign in first, then come straight back here.