How many specimens do I need? Sampling error in geometric morphometrics: testing the sensitivity of means and variances in simple randomized selection experiments

One of the most basic but problematic issues in modern morphometrics is how many specimens one needs to achieve accuracy in samples. Indeed, this is one of the most regularly posed questions in introductory courses. There is no simple and certainly no absolute answer to this question. However, there are a number of techniques for exploring the effect of sampling, and our aim is to provide an example of how this might function in a simplified but informative way. Thus, using resampling methods and sensitivity analyses based on randomized subsamples, we assessed sampling error in horse teeth from several modern and fossil populations. Centroid size and shape of an upper premolar (PM2) were captured using Procrustes geometric morphometrics. Means and variances (using three different statistics for shape variance) were estimated, as well as their confidence intervals. Also, the largest population sample was randomly split into progressively smaller subsamples to assess how reducing sample size affects statistical parameters. Results indicate that mean centroid size is highly accurate; even when sample size is small, errors are generally considerably smaller than differences among populations. In contrast, mean shape estimation requires large samples of tens of specimens (ca. >20), although this requirement may be less stringent when variance in a population is very small (e.g. populations that underwent strong genetic bottlenecks). Variance in either centroid size or shape can be highly inaccurate in small samples, to the point that sampling error makes it as variable as differences among spatially and chronologically well-separated populations, including two which are highly distinctive as a consequence of strong artificial selection. Likely, centroid size and shape variance require no <15–20 specimens to achieve a reasonable degree of accuracy. Results from the simplified sensitivity analysis were largely congruent with the pattern suggested by bootstrapped confidence intervals, as well as with the observations of a previous study on African monkeys. The analyses we performed, especially the sensitivity assessment, are simple and do not require much time or computational effort; however, they do necessitate that at least one sample is large (50 or more specimens). If this type of analyses became more common in geometric morphometrics, it could provide an effective tool for the preliminarily exploration of the effect of sampling on results and therefore assist in assessing their robustness. Finally, as the use of sensitivity studies increases, the present case could form part of a set of examples that allow us to better understand and estimate what a desirable sample size might be, depending on the scientific question, type of data and taxonomic level under investigation.