Review of: T.N.T.—Tree Analysis Using New Technology. Version 1.0, by P. Goloboff, J. S. Farris and K. Nixon. Available from the authors and from http://www.zmuc.dk/public/phylogeny

TNT, Tree Analysis Using New Technology, is a collaborative project by Steve Farris, author of Hennig86, Pablo Goloboff, author of PeeWee and Nona, and Kevin Nixon, author of Winclada, and can therefore be considered as the successor to each of these programs. From Hennig86 it inherits the terse style and the general format of the datafiles, from PeeWee/Nona the general tendency to consider the search for optimal trees as something that can and must be directed by the user, and from Winclada, to a large extent, the graphical interface for the display of trees and character optimizations (though not the data-editor). Compared to its predecessors Nona and PeeWee, TNT has an impressive array of new features. Some of these are based on the integration of features so far available in separate programs only, some are entirely new, and some are more gradual improvements on existing features. It is obvious that the research interests of the authors have to a large degree, guided the selection of new features. It should therefore not be a surprise that TNT does not include Maximum likelihood as an optimality criterion, does neither Neighbor-joining nor any other phenetic clustering, and there are some other conspicuously absent features, such as support for third-position characters as a separate partition or an Independent Length Difference test. Among the features that TNT does support, two are immediately obvious. One is the new windowed interface, which does away with the need to remember syntax for commands, parameters and options, and allows the examination and manipulation of trees and character mapping. The other consists of the improvements in tree space exploration, under the heading “New Technology”. These improvements allow the analysis, in a reasonable time, of datasets far larger than was previously possible. In addition, there are a number of other features integrated into TNT which were absent from the predecessor programs, or available only by way of macro-instruction files. TNT finds one of its major justifications in its capability to handle large datasets. Investigators facing the more than astronomically large tree spaces for large datasets have been increasingly relinquishing the search for shortest trees in favor of trees composed of well-supported nodes (Farris et al., 1996; see also, e.g., Salamin et al., 2003; Freudenstein et al., 2004), produced by methods such as parsimony jackknifing or quick bootstrap. Alternatively, the search for shortest trees has been stubbornly pursued, aided by algorithmic improvements such as those described by Nixon (1999) and Goloboff (1999). Most of the improvement in efficiency of these methods is due to the abandonment of the principle that each round of branch swapping should start on a tree built from scratch—these methods explore tree space with fewer starting points, but wider exploration than can be achieved by simple branch swapping, and thus they avoid getting stuck on a single large but suboptimal island. With TNT, the most efficient implementations of many of these approaches are available for the first time, in an easy to handle, user-friendly and compact package. A major improvement has also been made in the scripting language. TNT contains a fully developed programming language, which, apart from the standard TNT commands, gives access to a large number of internal variables used in branch swapping, tree evaluation, tree comparison and character optimization. It allows manipulation of arrays of up to five dimensions, virtually unlimited numbers of user variables, and recursive calls. The macro language is in fact the core of the TNT program, and allows extensions of its possibilities limited only by the user's programming capabilities. On the downside, of course, it is not an easy language to learn, but to users with any experience in programming it should not be a major obstacle. Sample files with macro instructions are distributed with TNT, which may serve to give some idea of the power of this language and of the way that it can be applied. Another novelty is that from within TNT, data entry and editing is possible. This does not takes place in a spreadsheet-like environment (as in Winclada), but by listing all character states for a selected taxon, or all taxa with a selected character state. In principle, it is even possible to start with an empty data matrix and to enter all data without leaving TNT, but this option is primarily useful to lookup and correct data entry errors when they become apparent in the course of an analysis. I found the option highly convenient, especially for the possibility of having a quick look at the distribution of a particular character state, but not really a replacement for dedicated spreadsheet-like data editors. When TNT starts, it opens as a single window, with a menu bar, a configurable toolbar, and a status indication at the bottom. The window itself serves as the display buffer. In standard mode, it echoes commands and results, and can be used to list trees in a character-based display. In addition, a command-line can be opened at the bottom of the window—but as this hides the lower part of the display, it is often easier to open it only when it is needed. Results are primarily written to the display buffer, and it is the user's responsibility to ensure that they are also saved to an output file. From the menu bar, submenus and windows open in a way that sometimes appears a little erratic, but that one gets quickly accustomed to. More intimidating is the way that some of the windows are crammed with text- and checkboxes; but after all, most of the options and parameters must have a place somewhere in a menu. Despite the appearance of a standard Windows interface, TNT partly uses its own conventions, which differ occasionally from the standard Windows conventions. For example, checking a check-box or radio button to indicate the use of a selection of taxa or characters will often immediately bring up a selection window, but in other cases, there may be a separate selection button to do this. In following its own conventions, TNT is not always consistent. Thus, it is best not to look for deeper meanings behind the different appearances that windows may take—the presence or absence of borders or title bars in windows is erratic rather than dependent on function. and some of the deviations from Windows standards are more annoying: the function of the tool buttons is not displayed as a standard “tooltip”-text, but appears in the status line, and then only when the button is pressed. It takes some agility not to execute the button after having pressed it in order to find out its function. Despite the presence of a menu-driven mode, the command-driven mode has been retained and is available both to procedure files and by way of a command line accessible from the graphical interface. Using a procedure file is the easiest way to enter data, trees and other complicated commands. All search procedures can be similarly run from a procedure file, but here the use of the menu interface is easier. The menu interface allows (as far as I have been able to ascertain) access to very nearly the same possibilities as the command line, and it is only rarely the case that some operations are more easily performed using commands. Setting constraints is the one example I found which can be performed with more flexibility using commands than using menus. In addition, there is a “batch mode” which stores and executes sequences of the menu choices. However, this batch mode does not allow for interactive menu choices, and can therefore only be used to automate relatively simple tasks. Most users who are intimidated by command-driven programs will be grateful for the presence of the menus. On the other hand, users who are accustomed to batch mode operations will not lose this ability. TNT accepts data in a format that is recognizably derived from Hennig86/Nona format, but is only partly backwards compatible, due to various modifications. These include a number of major improvements, such as the ability to read data in interleaved format, or data of mixed type, and some relatively minor ones, such as the inclusion of a separate command specifying the type of data, or the allowing of taxon names of up to 32 characters, and characters of up to 32 states. Together, the modifications are sufficient to disable the reading of most existing Hennig86 or Nona files. From Hennig86 and Nona, TNT inherits the still somewhat annoying habit of starting the numbering of most entities with 0. While I appreciate how this appeals to programmers, it is not the way most users think about numbers, and the best one can say about it is that one gets used to it. Confusingly, TNT does not consistently follow this convention in the case of numbering data blocks, where the number 0 is reserved and the numbering of the separate blocks starts with 1. In addition to ordered or unordered multistate characters, TNT accepts Step matrix (Sankoff) characters, and even offers the possibility of entering these in the shape of a character-state tree, either in a procedure file or using the menus. This is by far the most intuitive way that this type of data can be represented and entered, and this facility will no doubt stimulate the use of step matrices. For entering data, there is also a basic compatibility with the Nexus-format, which means that TNT can both read and write plain-vanilla Nexus files. So far, this only works for very simple files, and many of the elements that Nexus is intended to handle, such as comments interspersed with the data, will lead to error messages. Thus, although it is possible to present data in a way that allows an exchange of data and results between TNT and Nexus-based programs, this is not particularly easy, and requires careful considerations of the limitations of the programs involved. As in practice, all existing data files must be revised before they can be read by TNT, a list of the most important differences between the data format required by TNT and Hennig86, Nona and Nexus formats would have been a welcome addition to the documentation. Procedure files containing instructions for analysis must in all cases undergo major revision, as most of the commands used for performing an analysis have changed from their predecessors. Procedure files can only be run unattended when some elements in the menu interface (such as confirmation windows) are disabled. TNT implements three basic tree search strategies. First, it can do an exhaustive branch-and-bound search, or implicit enumeration. As we have become used to acknowledging, this option is impractical for datasets of over 20–30 taxa, but it is still there. Secondly, it can do traditional heuristic searches. For neither of these two strategies, does the TNT approach differ dramatically from those of other programs. The algorithms for implicit enumeration are the same as those used in Hennig86 or Nona, and heuristic searches are performed using multiple random addition sequences followed by branch swapping. The inclusion of new tree search methods has not made these two methods obsolete—for smallish datasets, an exhaustive search still is the only method that actually guarantees finding the shortest trees, and for the trees found using the newly introduced methods, traditional branch-swapping is still the best way to find more equally parsimonious trees. Under the catch-all term “New Technology”, search methods are included that have been developed to search for shorter trees in more efficient ways. It is in the implementation of these methods that a real quantum leap in computational speed has been made. The methods incorporated are sectorial search, ratchet (although in a somewhat different implementation than in Nixon's original description), tree drifting, and tree fusing. All of these methods can be used either singly or in combination, or they can be carried out as a “driven search”, where a driver program tracks the results of the analysis and decides on or adapts the next step to be performed. It is possible to set this driver to different levels of “aggressiveness”, which will change the relevant settings in a coordinated way. To illustrate the power of this New Technology, the distribution package includes the dataset first analyzed by Chase et al. (1993), and later reanalyzed by Rice et al. (1997). In the form distributed (“zilla”) it consists of 500 taxa and 759 characters. Using the “driven” search with standard settings, I found trees of length 16 220 (the optimal trees reported by Rice et al.) for this Zilla dataset within 2 min—and length of 16 218 (reported to be the shortest length for this dataset) in the next minute of running, but some luck may have been involved here: the next tree of length 16218 was not found for another 5 min. In another case, with a less “clean” dataset of 172 taxa and 69 morphological (often multistate) characters, I found that with a New Technology search, better trees than result from a multiple addition sequence are often quickly found, but that continued searches would lead to a slow, gradual and continuous improvement on these first trees. Thus, the New Technology methods appear to be very effective in finding short trees in what most investigators will be inclined to regard as a reasonable time. However, by their nature, they are not very effective in indicating the relative optimality of this tree. There is no indication whether the optimal trees represent single most parsimonious trees, or are trees from a large set of equally parsimonious trees. TNT offers three ways of solving this problem. First, one can always simply use the result of the New Technology search as the input tree for a round of standard branch swapping, either with or without the retention of suboptimal trees (for the computation of Bremer supports). Second, it is possible to run the “driven” search until a specified number of optimal trees have been found in independent replications—a very effective way of constructing consensus trees (Nixon, 1999). Lastly, there is the possibility of doing a “quick consensus” estimation using the method of Goloboff and Farris (2001). With poorly decisive datasets, I found all three methods very effective in removing spurious resolution from the initial consensus of the few trees returned form a single search. One of the problems that will arise for many users will center on the reproducibility of their results. Working with TNT, it quickly becomes clear that results of the New Technology search procedures can be difficult to reproduce. When using the “driven” search, it is impossible to predict at what time and at what stage during which round of one of the four basic methods a tree of optimal length will be found. When taking a more manual approach, it is equally difficult to tell what exact sequence of search commands is most effective and will lead to increasingly better trees. Users who, out of a concern for repeatability, have become accustomed to specifying search parameters in full detail when reporting the results of a search, face two options. Either they continue the practice of reporting all heuristic search settings, up to and including the for the random number that for a basic New Technology search, they would have to some useful be by doing Alternatively, they can simply that New Technology searches were until the reported tree was it to to find better trees using other this may their about the of their The of course, to the of the Working with TNT made that it is to the exact parameters for a search, and that that is highly methods are and was and been so it clear that large datasets required heuristic search The best trees may be found with methods that may be but due to the of random in the search The only in the procedure that is is the used to the length of a tree. For those who that this is all and that the of of methods should be there is of course always the option to start any search with a specified for the random and to list all search parameters in One of the limitations of TNT is that it can only out parsimony analysis. It is not possible to the algorithmic improvements used in TNT to tree search under different optimality It possible to to characters. TNT accepts and it implements method with the that a function can be TNT offers options to and not consensus trees. However, it does not have an option (as in to the consensus to a To a consensus tree, it must saved to in which case it is to the tree after all the other trees, and there to be no simple way after that to the consensus and it to a file using only the menu options. This appears to be one of the very few where useful is more easily by using commands than by using the menus. In addition to the standard ways of constructing consensus trees, TNT implements Goloboff and method of constructing This method from the way of but it is also possible to in the of the features of TNT is that any tree can be into its at the of a For branch TNT offers a number of of which or characters, but in different ways. from standard and analysis, the possibilities include and methods developed by Goloboff et al. to a number of of the standard it is to have these I they will as as the improvements in search strategies. The various improvements appear to have a minor on the support as far as I have been able to appear with support, taking into the of also with the support that can be with Using the graphical display options, characters can be trees using any of with the option to the characters on trees, or to the to all trees the on the For this character TNT is not limited to either under or optimization. with one option it all of the trees as with another all possible different While many users may be by the many possible character they will at be to consider a number of The results can be saved to the display in graphical format, to an in which case the tree can be or and can be made or I found this to be one of the most features of one of the elements to the and one may be by the of to the of I but the files are easily displayed in or Using a different graphical display, it is possible to and trees in ways. can be and by they can be the tree, and they can be for character support without leaving the graphical display. nodes or one can of taxa and so up of taxa to or which are not limited to nodes in a tree, but may also be TNT has in which with some basic and gives short of some of the more but does not a of all options and In addition, there is an internal basic accessible from the command line, which gives an listing of all the commands and their but does not to menu options. is included in an and distributed with The with TNT is thus over three different and many cases offers only with general such as that the time to find the best on the of the search. For a program with the of TNT, and with a as e.g., tree search, character and support, it of course, virtually impossible to write a manual or without it into an course on analysis. for the commands, many of which between 5 and users will need some for the for the parameters they have to is difficult to or is for most simply the settings will results. In order to really how one can use TNT and the results it it is to to the in which many of the procedures are first There is a very list of these in the out in more detail what TNT can do and how it works is often a of exploration, during which the commands and options under must be to simple files. I that in the we will see a in all the into a single documentation. the best way for users to and support is the TNT is a It the possibilities for parsimony analysis of large and difficult datasets with a quantum and it will therefore find its way to the users who are on large The presence of a really macro language will to those in or data However, TNT is not only useful for such but the for character and data editing will also to in the analysis of be it from a or a morphological It will find its way to the of many who are dependent on dedicated to analysis and character The of TNT may even be a for many of the to to set up a Windows to run it. TNT is distributed in a form as a is simple and When TNT, with a after which may use without or for of up to This is very and not easily The can be simply by entering the of the and a that is of the of should be to of the of is only and should not present an obstacle.