Small Sample Size Solutions/研究中小样本解决方案
香港科技大学
Textbook:Small Sample Size Solutions: A Guide for Applied Researchers and Practitioners
Author(s): Rens van de Schoot

Course description
Researchers often have difficulties collecting enough data to test their hypotheses, either because target groups are small (e.g., patients with severe burn injuries); data are sparse (e.g., rare diseases), hard to access (e.g., infants of drug-dependent mothers), or data collection entails prohibitive costs (e.g., fMRI, measuring phonological difficulties of babies); or the study participants come from a population that is prone to drop-out (e.g., because they are homeless or institutionalized). Such obs-tacles may result in data sets that are too small for the complexity of the statistical model needed to answer the research question. Researchers could reduce the required sample size for the analysis by simplifying their statistical models. However, this may leave the “true” research questions unanswered.
This coursebook is split into three parts:
Part I contains several chapters that describe and make use of Bayesian statis-
tics. Chapter 1 offers a gentle introduction to the main ingredients in Bayesian analyses and provides necessary information for understanding Bayesian parameter estimation and Bayes Factors. Chapter 2 offers a discussion of exchangeabil-ity and its role in the choice of sources of prior information in Bayesian analyses, which is relevant when combining datasets. Chapter 3 provides an extension of the When-to-Worry-and-How-to-Avoid-the-Misuse-of-Bayesian-Statistics (WAMBS) checklist, which is a 10-point checklist used to ensure optimal practices when applying Bayesian methods, extended to include prior and posterior predictive checking. Chapter 4 illustrates difficulties that can arise when implementing Bayesian solutions to a complex model and offers suggestions for avoiding these issues by making use of the effective sample size and divergent transitions. Chapter 5 provides a tutorial on Bayesian penalized regression for scenarios with a small sample size relative to the complexity of the stat-istical model by applying so-called “shrinkage priors” that shrink small effects towards zero while leaving substantial effects large.
Part II is composed of chapters on methods for analyzing data from a single participant. Chapter 6 introduces single-case experimental designs (n ¼ 1) and provides background information for analyzing a single-case experimental design (SCED) using unilevel design-based analysis. Chapter 7 discusses SCEDs in detail and provides an example of tests of effectiveness and change processes. Chapter 8 introduced a shiny app that allows researchers to supplement test scores of a single participant with teacher input or scores from other students in order to obtain a more accurate estimate of a given student’s ability. Chapter 9 presents a Bayesian method to evaluate hypotheses for each person in a sample and aggregate these results to find out whether a hypothesis holds for everyone in the sample, rather than for sample participants on average. Chapter 10 introduces a Bayes decision-making strategy for clinical trials, such that decisions can be made with smaller samples without increasing the risk of making an error.
Part III deals with complex hypotheses and models fit to small sample data. Chapter 11 provides examples and software for increasing power to detect mean differences by testing informative hypotheses within the framework of constrained stat-istical inference. Chapter 12 discusses several Bayesian methods for evaluating whether a finding was replicated across studies, which is extremely important in small sample research. Chapter 13 introduces software based on a machine-learning approach for identifying relevant moderators in meta-analysis. Chapter 14 provides an overview of the psychometric and model estimation benefits of parceling, and discusses how parcels can be particularly beneficial for small sample research. Chapter 15 offers an in-depth discussion of issues and potential solutions for multilevel models fit to small samples, from both frequentist and Bayesian perspectives. Chapter 16 describes several potential solutions for point estimation in structural equation models, including penalized likelihood estimation, a method based on model-implied instrumental variables, two-step estimation, and factor score regression. Chapter 17 compares, by means of a simulation study, two-step modeling, factor score regression, maximum likelihood, and Bayesian estimation with three prior spe-cifications for latent variable regression analysis with small samples. Finally, Chapter 18 offers a number of unique conclusions regarding data analysis with small samples.
The S4 Conference is a reoccurring event, and the research on optimal solutions to small sample size issues is ongoing. This coursebook represents a much-needed collection of currently available solutions, and we hope that it aids applied researchers in their endeavors and inspires methodological researchers to expand the field of small sample size solutions. We would like to thank all contributors for sharing their work, and give a special thanks to Evelien Schat and Gerbrich Ferdinands for their assistance with compiling this book. We hope to meet you, reader of this book, at our next conference.