Every Rule in the Setup Is a Promise About How Often You Are Wrong

You'll learnwhy both hypotheses are claims about the parameter and never about the sample, where the alternative's direction comes from, why the conditions and the standard error use the hypothesized proportion, what the significance level is committing to — and what each of those rules costs in false alarms when it is broken after the data arrive.

A company says 40% of its customers renew. A manager thinks it is lower, surveys 250 of them, and 88 renew — a sample proportion of 0.352. Setting up the test means writing four things down before any of that arithmetic happens, and those four rules sound like manners. This page prices them instead. Under a claim that is exactly true, a one-sided test at the 5% level cries wolf 5.24% of the time, as advertised; let the sample decide which tail to use and the same test cries wolf 10.64% of the time, under the same 5% label. And this survey's p-value is 0.0607, which sits between the two most common levels, so a level chosen afterwards picks the verdict by itself. Twelve short steps; press Next for each one.

12 STEPS · 6 QUICK CHECKS · HYPOTHESES ABOUT p · THE DIRECTION · CONDITIONS WITH p0 · ALPHA IN ADVANCE · v2

The claim's own curve, the region declared in advance, and one survey that fell short.

Before you start

What you're looking at

Every survey of 250 customers the company's claim could produce, if that claim were exactly true. The one real survey is the amber rule.

The question

A setup is four decisions made before the data are read. What does each one cost when it is made afterwards instead?

Step 1 / 12