A community-maintained standard library of population genetic models

1While population genetics has always used statistical methods to make inferences from data, the degree of sophistication of the questions, models, data, and computational approaches used have all increased over the past two decades. Currently, there exist a myriad of computational methods that can infer the histories of populations, the distribution of fitness effects, recombination rates, and the extent of positive selection in genome sequence data. While these methods have undoubtedly increased our understanding of genetic and evolutionary processes, very little has been done to systematically benchmark the quality of these inferences or their robustness to deviations from their underlying assumptions. As large databases of population genetic variation begin to be used to inform public health procedures, the accuracy and quality of these inferences is becoming ever more important.

2Assessing the accuracy of inference methods for population genetics is challenging in large part because the ‘ground-truth’ in question generally comes not from direct empirical observations, as the relevant historical processes can rarely be observed, but instead from simulations. Population genetic simulations are therefore critically important to the field, yet there has been no systematic attempt to establish community standards or best practices for executing them. Instead, the general modus operandi to date has been for individual groups to validate their own methods using simulations coded from scratch. Often these simulations are more useful to showcase a novel method than to rigorously compare it with competing methods. Moreover, this situation results in a great deal of duplicated effort, and contributes to decreased reproducibility and transparency across the entire field. It is also a barrier to entry to the field, because new researchers can struggle with the many steps involved in implementing a state-of-the-art population genetics simulation, including identifying appropriate demographic models from the literature, translating them into input for a simulator, and choosing appropriate values for key population genetic parameters, such as the mutation and recombination rates.

3A related issue is that it has been challenging to assess the degree to which modeling assumptions and choices of data summaries can affect population genetic inferences. Standardized simulations would enable these questions to be systematically examined. Importantly, there are clear examples of different methods yielding fundamentally different conclusions. For example, Markovian coalescent methods applied to human genomes have suggested large ancient (> 100,000 years ago) ancestral population sizes and bottlenecks that have not been detected by other methods based on allele frequency spectra (see ). These distinct methods differ in how they model, summarize, and optimize fit to genetic variation data, suggesting that such design choices can greatly affect the performance of the inference. Furthermore, some methods are likely to perform better than others under certain scenarios, but researchers lack principled guidelines for selecting the best method for addressing their particular questions. The need for guidance from simulated data will only increase as researchers seek to apply population genetic methods to a growing collection of non-model taxa.

4For these reasons, we have generated a standardized, community-driven resource for simulating published demographic models from a number of popular study systems. This resource, which we call stdpopsim, makes running realistic simulations for population genetic analysis a simple matter of choosing pre-implemented models from a community-maintained catalog. The stdpopsim catalog currently contains six species: humans, Pongo abelii, Canis familiaris, Drosophila melanogaster, Arabidopsis thaliana, and Escherichia coli. For each species, the catalog contains curated information on our current understanding of the physical organization of its genome, inferred genetic maps, population-level parameters (e.g. mutation rate and generation time estimates), and published demographic models. These models and parameters are meant to represent the field’s current understanding, and we intend for this resource to evolve as new results become available, and other existing models are added to stdpopsim by the community. We have implemented both a command line interface and a simple Python API that can be used to simulate genomic data from a choice of organism, genetic map, chromosome, and demographic history. In this way, stdpopsim will lower the barrier to high-quality simulation for exploratory analyses, enable rigorous evaluation of population genetic software, and contribute to increased reliability of population genetic inferences.

5The stdpopsim library has been developed by the PopSim Consortium using a distributed open source model, with strong procedures in place to continue its growth and maintain quality. Importantly, we developed rigorous quality control methods to ensure that we have correctly implemented the models as described in their original publication and provided documented methods for others to contribute new models. We invite new collaborators to join our community: those interested should visit our developer documentation at https://stdpopsim.readthedocs.io/en/latest/development.html. Below we describe the resource and give examples of how it can be used to benchmark demographic inference methods.

6The stdpopsim library is a community-maintained collection of empirical genome data and population genetics simulation models, illustrated in Figure 1. The package (https://github.com/popsim-consortium/stdpopsim) centers on a catalog of genomic information and demographic models for a growing list of species (Figure 1A), and software resources to facilitate efficient simulations (Figure 1B–C). Given the genome data and simulation model descriptions defined within the library, it is straightforward to run standardized simulations across a range of organisms. Stdpopsim has a Python API and a user-friendly command line interface, allowing users with minimal experience direct access to state-of-the-art simulations. Simulations are output in the ‘succinct tree sequence’ format, which contains complete genealogical information about the simulated samples, is extremely compact, and can be processed efficiently using the tskit library. The tree sequence format could also be converted to other formats (e.g., VCF) by the user if desired.

7The central feature of stdpopsim is the species catalog, a systematic organization of the key quantitative data needed to simulate a given species. Data are currently available for humans, P. abelii, C. familiaris, D. melanogaster, A. thaliana, and E. coli. A species definition consists of two key elements. Firstly, the library defines some basic information about our current understanding of each species’ genome, including information about chromosome lengths, average mutation rate estimates, and generation times. We also provide access to detailed empirical information such as inferred genetic maps, which model observed heterogeneity in recombination rate along chromosomes. Such maps are often large, so we do not distribute them directly with the software, but make them available for download in a standard format. When a simulation using such a map is requested by the user, stdpopsim will transparently download the map data into a local cache, where it can be quickly retrieved for subsequent simulations. In the initial version of stdpopsim, we support the HapMapII and deCODE genetic maps for humans; the maps for P. abelii; the map for C. familiaris; the map for A. thaliana; and the map for D. melanogaster. Adding further maps to the library is straightforward. The second key element of a species description within stdpopsim is a set of carefully curated population genetic model descriptions from the literature, which allow simulation under specific historical scenarios that have been fit to present-day patterns of genetic variation (see the Materials and methods for a description of the community development and quality-control process for these models.)

8The current demographic models in the stdpopsim catalog are shown in Table 1. Homo sapiens currently has the richest selection of population models. These include: a simplified version of the model with only the African population specified (expansion from the ancestral population and recent growth; Africa_1T12); the three-population model of, which specifies the out-of-Africa bottleneck as well as the subsequent divergence of the European and Asian populations (OutOfAfrica_3G09); the two-population variant of the Gutenkunst et al. model, which does not include Asian populations but more explicitly models recent rapid human population growth in Europe (OutOfAfrica_2T12); the admixture model for American populations, which specifies ancestral African, European, and Asian population components (AmericanAdmixture_4B11); a three-population out-of-Africa model from, which includes archaic admixture (OutOfAfricaArchaicAdmixture_5R19); a complex model of ancient Eurasian admixture from (AncientEurasia_9K19); and a synthetic model of oscillating population size from (Zigzag_1S14).

9For D. melanogaster, we have implemented the three-epoch model estimated by from an African sample (African3Epoch_1S16), as well as the out-of-Africa divergence and associated bottleneck model of, which jointly models African and European populations (OutOfAfrica_2L06). For A. thaliana, we implemented the model in inferred using MSMC. This model includes a continuous change in population size over time, rather than pre-specified epochs of different population sizes (SouthMiddleAtlas_1D17). We have also implemented a two-epoch and a three-epoch model estimated from African samples of A. thaliana in (African2Epoch_1H18 and African3Epoch_1H18).

10In addition to organism-specific models, stdpopsim also includes a generic piecewise constant size model and isolation with migration (IM) model which can be used with any genome and genetic map. Together, these models contain many features believed to affect observed patterns of polymorphism (e.g. bottlenecks, population growth, admixture) and therefore provide useful benchmarks for method development.

11To guarantee reproducibility, we have standardized naming conventions for species, genetic maps, and demographic models that will enable long-term stability of unique identifiers used throughout stdpopsim, as described in our documentation (https://stdpopsim.readthedocs.io/en/latest/development.html#naming-conventions).

12Currently, stdpopsim uses the msprime coalescent simulator as the default simulation engine. Coalescent simulations, while highly efficient, are limited in their ability to model continuous geography or complex selection scenarios, such as recurrent sweeps and background selection. For these reasons, we have also implemented the forward-time simulator, SLiM, as an alternative backend engine to stdpopsim, allowing for the simulation of processes that cannot be modeled under the coalescent. However, as forward-time simulators explicitly model all individuals in a population, simulating large population sizes can be highly demanding of computational resources. One common practice used to address this challenge is to simulate a smaller population, but to rescale resulting times, mutation rates, recombination rates, and selection coefficients so that the intensity of mutation, recombination, and allele frequency change due to selection per unit time remains the same (see the SLiM manual and ). Our implementation of the SLiM backend allows easy use of this rescaling through a single ‘scaling factor’ argument. Such down-scaled simulations are not completely equivalent to simulating all individuals in the population, and may lead to subtle differences, especially in the presence of selection. However, since many sequence-based measures of population diversity remain nearly unchanged when rescaling in this fashion, this practice is effective for many purposes and widely employed.

13We validated our implementation of the SLiM engine by comparing estimates of several population genetic summary statistics for neutral simulations generated by both SLiM and msprime. Examples of this validation for the AncientEurasia_9K19 model are shown in Appendix 1—figure 1 and Appendix 1—figure 2. For this model, down-scaling factors of up to 10 produce patterns of both diversity and linkage disequilibrium that are indistinguishable from those observed under the coalescent (i.e. msprime). Scaling down by a factor of 50 does appear to modify the distribution of these sequence statistics. Interestingly, the apparent difference between distributions is somewhat larger when simulating using a uniform recombination rate (Appendix 1—figure 2), likely due to the lower variation in the values of these statistics. Importantly, both comparisons validate the equivalence of SLiM and msprime when no down-scaling is applied. The results are also optimistic about the rescaling strategy to reduce computational burden, but the possible effects are not well-understood, so results relying on rescaled simulations should be carefully validated.

14The stdpopsim command-line interface, by default, outputs citation information for the models, genetic maps, and simulation engines used in any particular run. We hope that this feature will encourage users to appropriately acknowledge the resources used in published work, and encourage authors publishing demographic models to contribute to our ongoing community-driven development process. Together with the stdpopsim version number and the long-term stable identifiers for population models and genetic maps, this citation information will result in well-documented and reproducible simulation workflows. The individual tree sequence files produced by stdpopsim also contain complete provenance information including the command line arguments, operating system environment and versions of key libraries used.

15As an example of the utility of stdpopsim, we demonstrate how it can be easily used to perform a fair comparison of popular demographic inference methods. Although we present comparison of results from several methods, our aim at this stage is not to provide an exhaustive evaluation or ranking of these methods. Our hope is instead to demonstrate how stdpopsim will facilitate more detailed future explorations of the strengths and weaknesses of the numerous inference methods that are available to the population genetics community (see Discussion).

16We start by comparing popular methods for estimating population size histories of single populations and subsequently show simple examples of multi-population inference. To reproducibly evaluate and compare the performance of inference methods, we developed workflows using snakemake, available from https://github.com/popsim-consortium/analysis, that allow efficient computing in multicore or cluster environments. Our workflow generates replicates of chromosomes, producing population samples in each of a total of simulations for each demographic model. After simulation, the workflow prepares input files for each inference method by grouping all simulated chromosomes into a single file. Each file is then converted into an input file appropriate for each inference method (such that all inference methods run on the same simulation replicates). Each of the inference programs are then run in parallel, and finally, estimates of population size history from each program are plotted.

17For single-population demographic models, we compared MSMC, SMC++, and stairway plot on simulated genomes sampled from a single population, under several of the demographic models described above. However, these experiments raise the question of what to use as the ‘true’ population sizes in the case of multi-population models with migration. In particular, a simple single-population model that is fit to data simulated under a multi-population model, is not expected to recover the actual simulated population sizes because of model misspecification. Instead, we argue that the best one may expect in such a scenario is to infer a model that accurately reflects the coalescence time distribution of the simulated model. Under a multi-population model, the coalescence time distribution is influenced by migration between the target population and populations not analyzed in inference, as well as by the ancestral effective population sizes. The inverse coalescence rate is commonly interpreted as the effective population size, since these are equal in a single-population model with random mating. We thus analytically computed inverse coalescence rates in msprime for each simulated model, and used them as benchmarks for the ‘true’ effective population sizes. See the Appendix for a precise definition and description of the inverse coalescence rate computation.

18Figure 2 presents the results from simulations under OutOfAfricaArchaicAdmixture_5R19, a model of human migration out of Africa that includes archaic admixture, along with an empirical genetic map.In each column of this figure we show the inferred population size history (denoted ) from samples taken from each of the three extant populations in the model. In each row we show comparisons among the methods (including two sample sizes for MSMC). Blue lines show estimates from each of three replicate whole genome simulations, and black lines indicate the ‘true’ values depicted by the inverse coalescence rates (although in this specific model the inverse coalescence rates are very close to the simulated population sizes; Appendix 1—figure 3). While there is variation in accuracy among methods, populations, and individual replicates, the methods generally produce a good estimate of the true effective population sizes of the simulations, with inferred values mostly within a factor of two of the truth, and most methods inferring a bottleneck at approximately the correct time.

19Using stdpopsim, we can readily compare performance on this benchmark to that based on a different model of human history. In Appendix 1—figure 4, we show estimates of from simulations using the same physical and genetic maps, but from the OutOfAfrica_3G09 demographic model that does not include archaic admixture. Again we see that each of the methods is capturing relevant parts of the population history, although the accuracy varies across time. In comparing inferences between the models it is interesting to note that estimates for the CHB and CEU simulated populations are generally better across methods than estimates from the YRI simulated population.

20We can also see how well methods might do at recovering the population history of a constant-sized population, with human genome architecture and genetic map. We show results of such an experiment in Appendix 1—figure 5. All methods recover population size within a factor of two of the simulated values; however, SMC-based methods tend to infer sinusoidal patterns of population size even though no such change is present.

21As most method development for population genetics has been focused on human data, it is important to ask how such methods might perform in non-human genomes. Figure 3 shows parameter estimates from the African3Epoch_1S16 model, originally estimated from an African sample of D. melanogaster, and Appendix 1—figure 6 shows estimates from simulations of A. thaliana under the African2Epoch_1H18 model originally inferred by. In both cases, as with humans, we use stdpopsim to simulate replicate genomes using an empirically-derived genetic map, and try to infer back parameters of the simulation model. Accuracy is mixed among methods when doing inference on simulated data from these D. melanogaster and A. thaliana models, and generally worse than what we observe for simulations of the human genome.

22As stdpopsim implements multi-population demographic models, we also explored parameter estimation of population divergence parameters. In particular, we simulated data under multi-population models for humans and D. melanogaster and then inferred parameters using, fastsimcoal2, and smc++. For simplicity, we conducted inference in and fastsimcoal2 by fitting an isolation with migration (IM) model with constant population sizes and bi-directional migration. Our motivation for fitting this simple IM model was to mimic the typical approach of two population inference on empirical data, where the user is not aware of the ‘true’ underlying demography and the inference model is often misspecified. For human models with more than two populations (e.g. ) this limitation means that users are inferring parameters for a model that does not match the model from which the data were generated (Figure 4A and B). However, since the model used for inference also allows gene flow between populations, we directly compare estimated effective population sizes to the values used in simulations (black line in Figure 4C) and not the inverse coalescence rates.

23In Figure 4C, we show estimates of population sizes and divergence time, for each of the inference methods, using samples drawn from African and European populations simulated under the OutOfAfrica_3G09 model. Our results highlight many of the strengths and weaknesses of the different methods. For instance, the SFS-based approaches with simple IM models do not capture recent exponential growth in the CEU population, but do consistently recover the simulated YRI population size history. Moreover, these approaches allow migration rates to be estimated (Appendix 1—figure 7), and lead to more accurate inferences of divergence times. However, these migration rate estimates are somewhat biased. In contrast, smc++ is much better at capturing the recent exponential growth in the CEU population, though it consistently underestimates divergence times because it assumes no migration between populations (Figure 4C).

24Again, we can extend this analysis to other taxa and examine the performance of these methods for a two-population model of D. melanogaster. Appendix 1—figure 8 shows inference results using data simulated under the OutOfAfrica_2L06 model. This model includes an ancestral population in Africa from which a European population splits off following a bottleneck, with no post-divergence gene flow between the African and European population (Appendix 1—figure 8A). Here again, we find that and fastsimcoal2 infer more consistent histories, but they do not detect the brief bottleneck in Europe, due to the inference model not allowing for population size changes after the population split. In addition, and fastsimcoal2 both do reasonably well at correctly inferring the absence of migration (Appendix 1—figure 9). In contrast, the inferred demographic parameters from smc++ are more noisy, though in some cases better capture the short bottleneck in the European population.

25Although these results do not represent an exhaustive benchmarking, we have begun to highlight some of the strengths and weaknesses of these methods. Future work should build on these results and undertake more in-depth comparisons under a wider range of simulated demographic models.

26Here, we have described the first major product from the PopSim Consortium: the stdpopsim library. We have founded the Consortium with a number of specific goals in mind: standardization of simulation within the population genetics community, increased reproducibility and ease of use of complex simulations, community-based development and decision making guiding best practices in population genetics, and benchmarking of inference methods.

27The stdpopsim library allows for rigorous standardization of complex population genetic simulations. Population genetics, as a field, has yet to coalesce around a set of standards for the crucial task of method evaluation, which in our discipline hinges on simulation. In contrast, other fields such as structural biology and machine learning have a long track record of standardized method testing. We hope that our efforts represent the beginning of what will prove to be an equally longstanding and valuable tradition in population genetics.

28Besides being a resource for developers of computational methods, we aim for stdpopsim to be a resource for empirical researchers using genomic data. For instance, stdpopsim could be used in power analyses to determine adequate sample sizes, or in sanity checks to see if observed data (e.g. levels of divergence or the allele frequency spectrum) are roughly consistent with the hypothesized scenario. Currently, many studies would benefit from such simulation-based checks. However, there are major barriers to implementation, since individual research groups must reimplement complex, previously published demographic models, a task made especially daunting by additional layers of realism (e.g. recombination maps).

29We have illustrated in this paper how stdpopsim can be used for direct comparisons of inferential methods on a common set of simulations. Our benchmarking comparisons have been limited, but nevertheless reveal some informative features. For example, at the task of estimating population size histories for simulated human populations, we find that the sequence-based methods (MSMC and smc++) perform somewhat better overall—at least for moderate times in the past—than the site frequency spectrum-based method (stairway plot), which tends to over-estimate the sizes of oscillations (Figure 2 and Appendix 1—figure 4). In contrast, stairway plot outperforms the sequence-based methods on simulations of D. melanogaster or A. thaliana populations, in which linkage disequilibrium is reduced (Figure 3 and Appendix 1—figure 6). In simulations of two human populations (Figure 4), and fastsimcoal2 do reasonably well at reconstructing the simulated YRI history and estimating divergence times, but struggle with the more complex simulated CEU history, in large part because the methods assume constant population sizes. On the other hand, smc++ does not have the same restrictions on its inferred history, and as a result does much better with the CEU history but tends to underestimate divergence times due to the assumption of no migration. The results for the two-population D. melanogaster model (Appendix 1—figure 8) are generally similar. In these comparisons, fastsimcoal2 and perform almost identically, which is expected because they fit the same models to the same summaries of the data, differing only in how they calculate model expectations and optimize parameters.

30All methods for inferring demographic history have strengths and weaknesses (as recently reviewed by ). We compared inferences from simulated whole genome data, but many factors affect choice of methodology. Markovian coalescent methods (MSMC and smc++) require long contiguous stretches of sequence data. In contrast, frequency spectrum methods (stairway plot,, and fastsimcoal2) can use reduced-representation sequencing data, such as RADseq. and fastsimcoal2 require a pre-specified parametric model, unlike MSMC, smc++, and stairway plot. Using a parametric approach yields less noisy results, but a model that is too simple may not capture important demographic events (Figure 4 and Appendix 1—figure 8), and other forms of model misspecification may also produce undesirable behavior. From a software engineering perspective, methods also differ in their ease of installation and use. We hope our workflows will assist in the application of all the methods we have considered.

31Altogether, these preliminary experiments highlight the utility of stdpopsim for comparing a variety of inference methods on the same footing, under a variety of different demographic models. In addition, the ability of stdpopsim to generate data with and without significant features, such as a genetic map or population-size changes (e.g., Appendix 1—figure 5), allows investigation of the failure modes of popular methods. Moreover the comparison of methods across the various genome organizations, genetic maps, and demographic histories of different organisms, provides valuable information about how methods might perform on non-human systems. Finally, comparison of results across methods or simulation runs provides an estimate of inference uncertainty, analogous to parametric bootstrapping, especially when different methods are vulnerable to model misspecification in different ways.

32Stdpopsim is intended to be a fully open, community-developed project. Our implementations of genome representations and genetic maps for the some of the most common study systems in computational genetics—humans, Drosophila, and Arabidopsis (among others)—are only intended to be a starting point for future development. Researchers are invited to contribute to the resource by adding their organisms and models of choice. The stdpopsim resource is accompanied by clearly documented standard operating procedures that are intended to minimize barriers to entry for new developers. In this way, we expect the resource to expand and adapt to meet the evolving needs of the population genomics community.

33One of our goals is to engage research communities studying other taxa, so as to expand the resource to many more species. Although we have included demographic models and recombination maps, there are many biological processes that we do not model. Some of the additions that we are enthusiastic to add are: selection (including distributions of fitness effects, maps of functional elements, both single and recurrent hitchhiking events, and selection on polygenic traits), gene conversion, mutation models (rate heterogeneity), more realistic demography (overlapping generations, separate sexes, mortality/fecundity schedules), geographic population structure, and downstream aspects of data quality (genotyping and mapping error). Moreover, an in-depth investigation into the effects of population-size rescaling under many of the above scenarios is warranted, given our preliminary findings using neutral simulations (Appendix 1—figures 1 and 2). Some other important processes are more challenging to model with current simulation software, such as structural variation, changing recombination maps over time, transposable elements, and context-dependent mutation.

34We wish to emphasize that although the included demographic histories are some of the most widely used models for our current set of species, we anticipate the set of available models to expand as new methods and new modeling frameworks are developed. For instance, the current models all describe a small set of discrete, randomly mating populations, which are likely good approximations for deep-time population history, but may be less useful for methods describing dynamics of contemporary populations. Stdpopsim’s framework is sufficiently general that more realistic population models will be easily incorporated, as they are published. Additional aspects of the framework, such as genome builds, will also continue to change as improvements are made to our understanding of genome structure.

35As a consortium we have agreed to a standardized procedure for model inclusion into stdpopsim that allows for rigorous quality control. Imagine Developer A wants to introduce a new model into stdpopsim. Developer A implements the demographic model for the relevant organism along with clear documentation of the model parameters and populations. This model is submitted as a ‘pull request’, where it is evaluated by a reviewer and then included as ‘preliminary’, but is not linked to the online documentation nor the command line interface. Developer A submits a quality control (QC) issue, after which a second developer, Developer B (perhaps found by requesting review from the broader Consortium), then independently reimplements the model from the relevant primary sources and adds an automatic unit test for equality between the QC implementation and the preliminary production model. If the two implementations are equivalent, the original model is included in stdpopsim. If not, we move to an arbitration process whereby A and B first try to work out the details of what went wrong. If that fails, the original authors of the published model must be contacted to resolve ambiguities. Further details of our QC process can be found in our developer documentation (https://stdpopsim.readthedocs.io/en/latest/development.html).

36The possibility for error and the importance of careful qualty control was illustrated very clearly during our own development process: while carrying out the final revisions of this paper, we noticed that the OutOfAfrica_3G09 model had not gone through our QC process. The subsequent QC revealed that our implementation was in fact slightly wrong—migration rates had not been set to zero to the European population in the most ancient time period when there should have only been a single population. This error was propagated from the msprime documentation, where the model was presented as an illustrative example. A number of studies have been published using copies of this erroneous example code.

37To demonstrate the utility of stdpopsim we created Snakemake workflows that perform demographic inference on tree sequence output from our package using a few common software packages (see Appendix 1—figure 10 for an example workflow). Our choice of Snakemake allows complete reproducibility of the analyses shown, and all code is available from https://github.com/popsim-consortium/analysis.

38We performed two types of demographic inference. Our first task was to infer effective population size over time (denoted ). This was done using three software packages: stairway plot, which uses site frequency spectrum information only; MSMC, which is based on the sequentially Markovian coalescent (SMC), run with two different sample sizes; and smc++, which combines information from the site frequency spectrum with recombination information as in SMC-based methods. No attempt was made at trying to optimize the analysis from any particular software package, as our goal was not to benchmark performance of methods but instead show how such benchmarking could be easily done using the stdpopsim resource. In this spirit, we ran each software package as near to default parameters as possible. For stairway plot, we set the parameters numRuns = 1 and dimFactor = 5000. For smc++ we used the ‘estimate’ run mode to infer with all other parameters set to their default values. For MSMC, we used the --fixedRecombination option and used the default number of iterations.

39For the single-population task, we ran human (HomSap) simulations using a variety of models (see Table 1): OutOfAfricaArchaicAdmixture_5R19, OutOfAfrica_3G09, and a constant-sized generic model. Each simulation used the HapmapII_GRCh37 genetic map. For D. melanogaster we estimated from an African sample simulated under the DroMel, African3Epoch_1S16 model using the Comeron2012_dm6 map. Finally, we ran simulations of A. thaliana genomes using the AraTha African2Epoch_1H18 model under the Salome2012_TAIR7 map. For each model, three replicate whole genomes were simulated and the population size estimated from those data. In all cases, we set the sample size of the focal population to chromosomes.

40Following simulation, low-recombination portions of chromosomes were masked from the analysis in a manner that reflects the ‘accessible’ subset of sites used in empirical population genomic studies (e.g.; ). Specifically we masked all regions of 1 cM or greater in the lowest 5th percentile of the empirical distribution of recombination, regions which are nearly uniformly absent for empirical analysis. This approach to masking was chosen to prevent marginal trees with low or no recombination from biasing the comparisons of demographic inference methods. It should be noted that masking is not implemented within stdpopsim proper; tree sequences generated by stdpopsim are always raw and unmasked. This allows users the flexibility to implement masking approaches that are specific to their needs for downstream analysis.

41Our second task was to explore inference with two-population models using some of the multi-population demographic models implemented in stdpopsim. For HomSap, we used the OutOfAfrica_3G09 model with the HapmapII_GRCh37 genetic map, and for DroMel we used the OutOfAfrica_2L06 model with the Comeron2012_dm6 map. The HomSap model is a three population model (Africa, Europe, and Asia) including post-divergence migration and exponential growth (Figure 4C), whereas the DroMel model is a two population model (Africa and Europe) with no post-divergence migration and constant population sizes (Appendix 1—figure 8).

42To conduct inference on these models, we applied three commonly used methods:, fastsimcoal2, and smc++. As above, these methods were used generally with default settings and we did not attempt to optimize their performance or fit parameter-rich demographic models.

43For both and fastsimcoal2, we fit a two population isolation-with-migration (IM) model with constant population sizes. This IM model contains six parameters: the ancestral population size, the sizes of each population after the split, the divergence time, and two migration rate parameters. Importantly, this meant that for both species, the fitted model did not match the simulated model (Figure 4 and Appendix 1—figure 8). In the HomSap case, we therefore performed inference solely on the Africa and Europe populations, meaning that the Asia population functioned as a ‘ghost’ population that was ignored by our inference. To validate our inference approach, we also conducted inference on a generic IM model that was identical to the model used for inference (Appendix 1—figure 11).

44From HomSap simulations, we took 20 whole genome samples each from the Europe and Africa populations from each replicate. Runtimes of DroMel simulations were prohibitively slow when simulating whole genomes with the Comeron2012_dm6 map due to large effective population sizes leading to high effective recombination rates. For this reason, we present only data from 50 samples of a 3 MB region of chromosome 2R from simulations under OutOfAfrica_2L06. For the generic IM simulations, we used the HomSap genome along with the HapmapII_GRCh37 genetic map and sampled 20 individuals from each population.

45Following simulation, we output tree sequences and masked low-recombination regions using the same approach described for the single population workflow above. We converted tree sequences into a two-dimensional site frequency spectrum for all chromosomes in the appropriate format for and fastsimcoal2. For each simulation replicate, we performed 10 runs of and fastsimcoal2, checking to ensure that each method reached convergence.

46Detailed settings for and fastsimcoal2 can be found in the Snakefile on our git repository (https://github.com/popsim-consortium/analysis). Estimates from the highest log-likelihood (out of 10 runs) for each simulation replicate are shown in Figure 4C and Appendix 1—figure 8C.

47For smc++, we converted the tree sequences into VCF format and performed inference with default settings. Importantly, smc++ assumes no migration post-divergence, deviating from the simulated model. However, because smc++ allows for continuous population size changes, it is better equipped to capture many of the more complex aspects of the simulated demographic models (e.g. exponential growth).

48To visualize our results, we plotted the inferred population size trajectories for each simulation replicate alongside the simulated population sizes (Figure 4C and Appendix 1—figure 8C). Here, unlike the single-population workflow, we compare our inferred population sizes only to the simulated population sizes and not the inverse coalescence rates.

49The stdpopsim package is available for download on the Python Package Index: https://pypi.org/project/stdpopsim/. Documentation for the project can be found here: https://stdpopsim.readthedocs.io/en/latest/.