Specifically, we extend their simulations to show that the SPRT and SBFT are procedurally and mathematically similar, and that their relative efficiency depends on the exact model specification and the true population parameters. Here, we demonstrate that the simulation conditions used by Schnuerch and Erdfelder ( 2020) constitute an extreme case within an otherwise more nuanced relationship between the two hypothesis testing procedures. Our paper will use the t-test as an example as well, but our results apply to the broader class of SPRTs and SBFTs. Recently, there has been a strong focus on developing, comparing, and promoting sequential testing procedures for the independent-samples t-test, one of the most commonly used statistical tests in psychological research (Wetzels et al., 2011). For example, Bayesian sequential testing has been applied in developmental psychology in experiments on early word learning (Mani et al., 2020), or in cognitive psychology in experiments on learning and decision making (Perquin et al., 2020 Stojić et al., 2020). With the increasing uptake of Bayesian statistics in psychology in the past years, SBFTs have become more popular as well. The SPRT has also been used to study speeded decision making in humans and other animals (Luce, 1986 Townsend & Ashby, 1983 Bogacz et al., 2006 Ratcliff, 1978 Purcell et al., 2010 Milosavljevic et al., 2010), and is the basis for modern models of human decision making such as the Drift Diffusion Model (Bogacz et al., 2006 Griffith et al., 2021). Seminal work by Reckase ( 1983) combined the SPRT with item response theory, and has been the basis for more recent methodological developments in the field of computerized classification testing (e.g., Lin and Spray 2000 Eggen & Straetmans 2000 Eggen 2011 Finkelman 2008). In psychological assessment, Ferguson designed a sequential mastery test using the SPRT methodology as early as 1969.
In some disciplines of psychology, the SPRT has already enjoyed great popularity for many years. We provide guidance for navigating these design decisions based on individual preferences and simulation-based design analyses. Our simulations indicate that when deciding on a sequential design within a unified sequential testing framework, researchers need to balance the needs of test efficiency, robustness against model misspecification, and appropriate uncertainty quantification. We demonstrate that the two methods use the same mechanisms for evidence monitoring and error control, and that differences in efficiency between the methods depend on the exact specification of the statistical models involved, as well as on the population truth. We show that although the two methods have different philosophical roots, they share many similarities and can even be mathematically regarded as two instances of an overarching hypothesis testing framework. Here, we compare two sequential hypothesis testing procedures that have recently been proposed for use in psychological research: Sequential Probability Ratio Test (SPRT Psychological Methods, 25(2), 206–226, 2020) and the Sequential Bayes Factor Test (SBFT Psychological Methods, 22(2), 322–339, 2017). As soon as sufficient information has been obtained, data collection is terminated.
In a sequential hypothesis test, the analyst checks at multiple steps during data collection whether sufficient evidence has accrued to make a decision about the tested hypotheses.
0 kommentar(er)
