Erin Gabriel, IMM
A boundary optimized rejection region test for the two-sample binomial problem
Testing the equality of 2 proportions for a control group versus a treatment group is a well-researched statistical problem. In some settings, there may be strong historical data that allow one to reliably expect that the control proportion is one, or nearly so. While Bayesian tests, one-sample tests or comparisons to historical controls could be used, none of these will rigorously control the type I error rate in the event the true control rate changes. In a recent work, I proposed an unconditional exact test that exploits the historical information while controlling the type I error rate. I sequentially construct a rejection region by first maximizing the rejection region in the space where all controls have an event, subject to the constraint that our type I error rate does not exceed α for any true event rate; then with any remaining α maximizes the additional rejection region in the space where one control avoids the event, and so on. When the true control event rate is one, this test is the most powerful nonrandomized test for all points in the alternative space.
When the true control event rate is nearly one, one can demonstrate that our test has equal or higher mean power, averaging over the alternative space, than a variety of well-known tests. A powerful example of this is a sample size of 8, comparison of 4 controls and 4 treated subjects, where our proposed test has higher power for a notable reason which I will explore.Contact person: Federica Laguzzi