Gotree/Goalign: toolkit and Go API to facilitate the development of phylogenetic workflows

1Increase in computer power and development of bioinformatics methods that handle very large datasets make it possible to perform phylogenetic analyses at an unprecedented scale. For example, it is common to perform phylogenetic studies involving several thousand trees or several thousand taxa (e.g. or). Such studies often present and run complex pipelines involving many steps and several tools and scripts, which makes them hard to describe and share. For example, the current COVID-19 pandemics and the diverse SARS-CoV-2 data analysis workflows that flourish demonstrate the need to manipulate sequences, alignments, and phylogenetic trees in an easy and automated way (e.g. https://github.com/roblanf/sarscov2phylo/).

2Figure 1 displays an example of a phylogenetic workflow inspired from and described in details in the Results section. The main objective of this workflow is to analyse a phylogenomic dataset to infer a species tree and compare it to a reference tree. It contains 19 steps among which the majority is not constituted by usual computer intensive tasks (e.g. tree inference and multiple sequence alignment), but rather by alignment and tree manipulations such as downloading, renaming, reformating, comparing, rerooting, annotating, etc. These tasks are (i) repetitive (found in many similar workflows), (ii) tedious to implement (many ways to do it) and (iii) error prone, which makes this workflow difficult to implement, describe and reproduce.

3That is why we think a common framework was needed to help implement such analyses in a concise and reproducible way.

4Several tools, either in the form of command line or APIs, already exist to manipulate phylogenetic trees and/or alignments. For example, Newick-utilities is a command line tool dedicated to phylogenetic tree manipulation in newick format. ETE toolkit is a Python API for the analysis and visualization of trees. Ape is a package dedicated to the analyses of phylogenetics and evolution in the R language. Buddy Suite is another python framework (command line) to manipulate phylogenetic trees and alignments. BIO++ is a C++ library for the analysis of sequences and trees. Lastly, Phyx is a command line tool written in C++ that performs phylogenetic analyses on trees and sequences. Some of these tools (e.g. ape, ETE toolkit, BIO++) are mainly developer oriented and may be difficult to use for non-programmer phylogenetic analysts who are not familiar with R, python or C++. Others are command line only (e.g. Newick-utilities), and may be of limited use for developers who want to use part of them in their own programs. They are all implemented either in C, C++, Python or R. To our knowledge, no phylogenetic tree toolkit currently exists, which mix alignment and tree manipulation, in a command line environment and via an API, and proposes a large diversity of commands. In particular, none exists for the Go programming language for which several bioinformatics libraries and tools already exist such as biogo and Vcfanno.

5In this context, we developed Gotree/Goalign, a toolkit dedicated to burdensome and repetitive phylogenetic tasks, which (i) consists of two user-friendly executables, gotree and goalign, integrating state of the art phylogenetic commands and requiring no programming skills to use, (ii) provides a set of chainable commands that are integrable in workflows (e.g. Nextflow or Snakemake), (iii) is straightforward to install via static binaries available for most platforms and architectures, and (iv) provides a public API accessible to developers wanting to manipulate phylogenetic trees and multiple sequence alignments in Go.

6The Gotree/Goalign toolkit is already used in several public workflows such as Grapevine (the phylogenetic pipeline for the COG-UK project https://github.com/COG-UK/grapevine), ARTIC-EBOV (The ARTIC Ebola virus phylogenetic analysis protocol https://github.com/artic-network/artic-ebov), KOVID-TREES-NF (a Nextflow workflow for SARS-CoV-2 phylogenetics https://github.com/MDU-PHL/kovid-trees-nf), bioconvert (a generic bioinformatic file format converter https://github.com/bioconvert/bioconvert), and in phylogenetic studies.

7Goalign implements more than 60 commands to manipulate sequences and multiple sequence alignments. A subset of these commands is given in Table 1.

8The most obvious (and maybe the most widely used) commands reformat an input sequence file into any supported format, which is essential to integrate different tools in the same analysis workflow. The supported input and output formats are: Fasta, Phylip, Clustal, Nexus, and tool specific formats (e.g. TNT and PaML input formats). It is worth noting that Goalign supports compressed input files (.gz,.bz and.xz) and remote files through http(s).

9A second kind of commands is designed to extract summary statistics about input alignments, such as the length, the number of sequences, the frequency of each character, the list of sequence names, etc.

10The third kind of commands aims at modifying input alignments (e.g. extract sequences or sites, clean, translate, rename, shuffle, mask, concatenate, append, etc.) or producing new alignments (build bootstrap alignments, align by codon, align pairwise, etc.).

11Finally, some commands are designed to compute distance matrices for nucleotidic and proteic alignments. To this aim, Goalign implements main nucleotidic evolutionary models (JC, K2P, F81, F84 and TN93) and amino-acid matrices (DAYHOFF, JTT, MtRev, LG, WAG). Distances between amino-acid sequences are computed using maximum likelihood as in PhyML and FastME.

12Gotree implements more than 60 commands dedicated to the manipulation of phylogenetic trees (see Table 1 for a list of a few of them).

13As for Goalign, Gotree reformatting commands are the first to be used in phylogenetic workflows and constitute the glue between all phylogenetic tools, which often support heterogeneous, specific formats. Gotree supports the following input and output formats: Newick, Nexus and PhyloXML. Moreover, Gotree supports compressed input files (.gz), and remote files from any urls or from dedicated servers (iTOL and Treebase, See Supp. Text 1 for examples of such commands).

14The second kind of commands implemented in Gotree produces summary statistics about input phylogenetic trees, such as the number of tips, number of branches, average branch length and branch support, sum of branch lengths, number of cherries, tree balance indices (Colless, Sackin), and patristic distance matrix.

15A third kind of commands modifies the input tree, e.g. modifying branch lengths and supports, collapsing branches by length or support, removing lists of tips, rerooting the tree in different ways, etc.

16Other commands generate new trees or new data on the trees, such as bootstrap support computations (Felsenstein’s Bootstrap Proportions and Transfer Bootstrap Expectation) or random tree generation using different models.

17The last kind of commands compares trees, for example comparing topologies (bipartition distance), or comparing tip names.

18Put together, these commands provide the user with a large panel of possibilities adapted to many different situations, a few of which are described in the Results section.

19The Gotree/Goalign toolkit is implemented in the Go (https://golang.org/) programming language, and thus takes advantage of (i) being available on major operating systems (Windows, Linux, MacOS), (ii) the richness of the Go standard library and the massive amount of available packages (http, hashmaps, compression, regex, channels, concurrency, gonum, etc.), (iii) the distributed nature of Go packages, providing phylogenetic packages easily accessible to any developers.

20It consists of two executables gotree and goalign providing all the commands in the same spirit as git or docker.

21All the commands are implemented in the Unix mindset, having one atomic command per task, reading data from the standard input and writing results to the standard output when possible.

22The Gotree/Goalign toolkit has been developed to be easily integrated in phylogenetic workflows, using main workflow managers such as Nextflow or Snakemake. Two main characteristics facilitate this behavior.

23First of all, Gotree/Goalign consists of a large set of command line tools that can be easily chained, and that avoid manual interventions as much as possible. This feature makes them easily usable in workflows as all commands are atomic, easily citable, describable and reproducible.

24Moreover, they are made available through three channels that are compatible with the diversity of use in workflow managers, i.e., multi-plateform executables, Docker and Singularity containers, and Bioconda packages.

25In this use case, we analyze a phylogenomic dataset inspired from, in which the authors analyze a set of 1730 genes from primates. They infer the species tree either from individual gene trees using ASTRAL III or from gene concatenation using maximum likelihood. Our use case is inspired from the concatenation study, inferring a species tree from available groups of primate orthologous proteins in OrthoDB after having mapped the RefSeq identifiers of the 1730 analyzed genes to their OrthoDB counterpart.

26The workflow is displayed in more detail in Figure 1 (code available at https://github.com/evolbioinfo/gotree_usecase and in Supp. Data 1), and consists of the following main steps (details of the tools and options are given in Supplementary Figure S1): (i) mapping RefSeq, HGNC and OrthoDB identifiers; (ii) retrieving each group of orthologous proteins in OrthoDB if they are present in at least 90% of the 25 primate species (Supp. Data 2) and in a single copy (this results in 1315 orthologous groups, listed in Supp. Data 3); (iii) for each orthologous group, downloading the sequences of all proteins and their associated metadata; (iv) renaming, cleaning and aligning sequences using MAFFT; (v) concatenating the alignments of the 1315 orthologous groups (this results in a single alignment of 674 089 amino-acids); (vi) cleaning the alignment using BMGE (this keeps 516 999 sites); (vii) inferring the species tree using IQ-TREE; (viii) downloading the NCBI taxonomy; (ix) comparing the inferred tree with the NCBI taxonomy and (x) uploading both trees to iTOL. Each step and their parameters are described in detail in Supplementary Figure S1.

27Since it involves many very different steps, this example is challenging to implement and to run fully automatically, without manual intervention. Apart from tree inference and multiple sequence alignment, most of the operations required to develop this workflow are available in the Gotree/Goalign toolkit and are straightforward to execute (Supplementary Figure S1). The implementation of the workflow, made of 19 steps and ∼350 lines of code, is fully homogeneous in terms of file formats and input/outputs. Furthermore, the ability to run the full analysis, from the input data to the output tree visualization, without manual intervention facilitates its description and reproducibility.

28The resulting primate phylogeny, given in Figure 2, is in almost complete agreement with the NCBI taxonomy, with 15 out of 16 branches of the NCBI taxonomy found in our tree. It is worth noting that the only conflicting branch (in red), grouping Callitrix and Aotus, has a lower bootstrap support than other branches and is also present in the tree inferred using maximum likelihood from gene concatenation in. In fact, our topology and theirs are identical.

29We developed the Gotree/Goalign toolkit to simplify the manipulation of phylogenetic trees and alignments, and to facilitate the development of reproducible phylogenetic workflows. Importantly, it is not a wrapper around major tree inference and multiple sequence alignment software, i.e. it does not take care of aligning sequences and inferring trees, but rather takes care of all the other complex, tedious and numerous tree and sequence operations that are necessary in phylogenetic workflows. The Gotree/Goalign toolkit is developed in Go and is easily installable on major operating systems and provides a public API usable in any Go project.

30It is already used in several projects, and we are confident that Gotree/Goalign will be able to build a community interested in adding new functionnalities.