tacg Version 4.3 User Guide

tacg is a character-based, command line tool for unix-like operating systems for pattern-matching in nucleic acids and performing some of the basic protein manipulations. It was originally designed for restriction enzyme analysis of DNA, but has been extended to other types of matching. It now handles degenerate sequence input in a variety of matching approaches, as well as patterns with errors, regular expressions and TRANSFAC-formatted matrices.

It was designed to be a grep for DNA and like the original grep, its capabilities have grown so that now the author has to keep calling up the help page to figure out which flags (now ~50) mean what. tacg is NOT a GUI application in any sense. However, it’s existance as a strictly command-line tool lends itself well to Webification and wrapping by various GUI tools and it is now distributed with a web interface form and a Perl CGI handler. Additionally, it can easily be integrated into editors that support shell commands (see Nedit below).

The entire feature set is described in more detail below, but it includes:

In short, it does a lot, costs a little, and is very fast.

tacg is available as a binary in deb form for Debian-based systems, and as a source tarball that should compile on any linux system or indeed anything with gcc. (tested on x86, AMD64, Mac OSX? AIX?, Solaris?).

If you are morally upright and that is reflected in your computing environment because you run a Debian flavor of Linux. Get the deb here

And install it: $ dpkg -i tacg-4.3.0.deb

Get the latest tarball here

or from the Sourceforge website

tacg relies on the standatrd unix conventions of redirection for input and output. You can redirect another program’s STDOUT to tacg via the pipe (|) character:

For wrapping by other programs, it also supports commandline input (--infile) and output (--outfile) file specifications.

tacg uses the GNU getopt routines to process input options, which means that you can abbrieviate the long options (--numstart=343) to the minimum number of characters that differentiate them from all other options (--nu=343). You can also pool single letter options that have no additional flags (-l -s can be abbrieviated -sl)

typing:

by itself will emit an informational message about how to get more information about using it.

Here’s an example:

Translation: Process the input file lamcg.gb transparently thru Jim Knight’s seqio routines and pipe the extracted sequence to tacg, returning the analysis:

NB: the single letter options -l -c -L could have been pooled as -lcL

tacg will take any ASCII data as input. If it isn’t a recognized format, it will assume that it’s raw sequence and will process it accordingly. You can also explicitly tell it that all input is to be considered sequence with the --raw flag. If this is the case, it will ignore anything that isn’t a,c,g, u, or t (or IUPAC degeneracies) and subject that sequence to analysis. Because of this, numbering, spacing, line width, etc, of the input file should not be a problem. The Web interface allows you to upload entire files of sequence for analysis. The use of Jim Knight’s SEQIO allows most ASCII sequence formats (see below) to be read and processed without further modification or editing.

However, it will NOT reliably accept binary file formats such as those from DNA Strider, MacVector, NBI, GeneWorks, etc. However, most commercial programs have an ASCII export function that should work. For the WWW interface, anything that you can cut and paste into the input window and looks like sequence ought to be fine. As mentioned above, the --raw option skips SEQIO and so reverts tacg to it’s former aggressive behavior of considering EVERYTHING sequence.

tacg can handle degeneracy in both input sequence and patterns being searched for. When it hits a degeneracy in the hexamer window, it switches to a more expensive matching approach until the degeneracy is cleared, whereupon it switches back to the faster approach. This allows tacg to accurately search sequence that is only slightly degenerate (such as many genomic sequences) essentially as fast as nondegenerate sequence. See the explanation of the -D option in the man page.

While most uses of tacg will involve analyses of single sequences at a time, it can process multiple sequences at a time and output stanzas prefixed by the sequence header. There is only one option currently that is designed for multiple sequence processing, the --idonly flag, below.

-i or --idonly {0|1*|2}

When analyzing multiple sequences at once, it can be more efficient to print the results only from those sequences which contain hits. If a sequence does not contain hits, you might not want the summary info which would usually accompany processing the sequence. The -i (or --idonly) flag allows better control of how much info gets printed per sequence:

tacg allows sequences to be subselected by specifying a beginning and end. If -b # (see below) is used, the indices shown in the Sites output correspond to the starting offset as 1, not the real starting number. For example, if there was a BamHI site at 200 in the complete sequence and you took a subsequence with -b50, that same BamHI site would be reported at 150, not 200. In the Linear Map option, you can have both the original numbering as well as the subsequence numbering listed.

-b offset -e offset

These flags simply allow you to specify the subsequence by starting offset and ending offset.

(-X or --extract b,e,[0|1)]

This is a very useful option for extracting sequences around a hit to examine it for related sequences, useful for examining genomic sequences for related transcription factor-binding sites. The output is FASTA-formatted, and so is ready to be fed to a multiple-alignment program such as clustalw.

(-f {0|1*})

Informs tacg whether the input sequence is meant to be considered linear (1-default) or circular (0). The practical result only impacts the Fragments Table and the Gel Map, as a circular sequence will result in one less fragment than a linear sequence.

NB: tacg is not bright enough to know that translations that cross the begin/end transition should be continued if the sequence is circular, so it will miss the ORFs that do this. If you are concerned about this, split your sequence in the middle and stick the begin/ends together and re-run the analysis.

tacg has 3 options to allow sequences to be flipped and flopped before analysis. This is handy when resolving questions about sequencing and orientation.

(--rev)

Reverses all input sequences before analysing them: tacg → gcat

(--comp)

Complements all input sequences before analysing them: tacg → atgc

(--revcomp)

Reverse-Complements all input sequences before analysing them: tacg → cgta

(-x Label,Label(=),Label..(,C))

Enzymes can be picked explicitly via the -x flag (-x NameA,NameB,NameC… where NameX is the case INsensitive name of the Enzyme or pattern you wish to use in the GCG-formatted REBASE file, either rebase.data or another specified by the -R flag (-R alt.rebase.file).

This option can also be used to simulate a multiple digest (as when you add 2 or more enzymes to a reaction for diagnostics or to generate a particular combination of ends. The format to request this action is to append a ,C to the list of enzymes: -x BanI,HindIII,NdeIII,C.

The -x option can also be used to do a crude AFLP analysis. If you append a = to one of the RE names, it will tag that RE as labelled and any fragment that contains an end generated by that RE will be noted.

you can select REs based on:

(-n #)

The magnitude of the recognition sequence depends on the number of defined bases that make up the site. Degenerate bases can also contribute:

For example: tgca=4, tgyrca=5, tgcnnngca=6, etc

(-o {0|3|5})

This refers to the stretch of unpaired bases at the border of the cut created by offset breaks in the phosphate backbone. Enzymes can leave:

(-m # ), (-M #)

This is pretty self explanatory, but.. Minimum indicates the minimum # of times a pattern HAS TO match before it’s included in the output (the default is 1). Maximum indicates the maximum # of times an enzyme CAN match before it’s excluded from the output (no default maximum). You can use both at same time as long as m < M)

(--cost #)

This flag requires that the extra, optional cost data be added to the rebase.data file. It is included with the rebase.data distributed with tacg, but not with ones supplied by NEB. See the entry describing the extended rebase file format

(--dam) (--dcm)

The flag --dam simulates cutting in the presence of Dam methylase (G`mA`TC). rebase.dam contains all REs that are Dam-sensitive. --dcm simulates cutting in the presence of Dcm methylase (C`mC`WGG). rebase.dcm contains all REs that are Dcm-sensitive. The flags can be used together.

(-R alt filename)

Using this option, you can select an alternate rebase file to use (and then select specific enzymes from it using the other methods of subselection as described above. This would allow you to collect related patterns into a file to search for at once, for example. This option is also used to specify alternate files for RE patterns in the same format as the rebase.data file and for TRANSFAC matrix patterns. Note that for internal logic reasons, the specification for files of regex patterns is different. See the -r flag for details.

Besides selecting pre-defined patterns from files such as rebase.data, you can also specify patterns at the commandline for both IUPAC patterns and regular expressions. TRANSFAC matrices are too complex to specify from the commandline, so must be edited into an appropriate file to use. However, see below for how to specify them from the commandline.

(-p Label,Pattern[,Err)]

The -p flag requires a name and a pattern and an optional error term. The 'Label` can be any combination of alphabetic characters up to 10 characters. The Pattern is a string of nucleic IUPAC characters (acgtnmkwsbdhv) up to 30 characters, and the optional Err term is an integer (⇐3) that specifies how many errors can be tolerated in the pattern and still count as a hit.

Any patterns that are entered in this manner are also written to a file named tacg.patterns in the current directory in a format suitable for entering into the rebase.data file.

(-r Label:RegexPat or FILE:RegexFile)

Regular Expressions can be entered from the commandline as well as read in from a file. To enter a regex from the commandline, the option flag requires a Label as an identifier and the regex itself in the form above. This option uses the pcre regex library so the regexes are perlish in flavor. As a small favor, tacg will translate IUPAC characters into their regex equivalents:

To read in a file of regex’s, the form is the same, except that the Label must be the special case FILE and the string following is not a regex, but the path to the specially formatted regex file.

Note that in both cases, the string following the -r must be single quoted to prevent the shell from mangling the expression inside.

-# %_Match_CutOff

While TRANSFAC matrices cannot be defined on the commandline, you can search with already defined ones using the -# flag (and possibly the -x flag if you want to search for particular ones).

The %_Match_CutOff value above is the percentage match cutoff that has to be met over the whole of the pattern. That is, the pattern has to be %_Match_CutOff % identical to the matrix to count as a hit. See the xman entry for an example.

The -x flag is used to select specific matrices from a file containing many entries. If no specific entries are given, ALL entries in the file are searched.

You can select among several matrix files with the -R flag like selecting alternative rebase files.

This type of pattern matching involves searching for two or more patterns that occur in a particular super-pattern. tacg allows 2 kinds of such searching for each of the 3 types of basic patterns (IUPAC, regex, matrix). The first I call Proximity matching which involves specifying the relationship of 2 patterns quite precisely. The second I call rules-based matching and involves specifying the logical and spatial relationship among many patterns, tho with less precision than with the Proximity matching.

-P rulestring

This is a specialized search in which you can very precisely specify the spacial relationship between 2 (and only 2) patterns - NameA and NameB in the example above. The NameA/B labels bracket the spatial relationship expression, an unavoidably baroque confection which can be deconstructed with some careful study:

The above-mentioned rulestring in this case looks like this: NameA,[+-][lg]Dist_Lo [-Dist_Hi],NameB, where NameA and NameB are the pattern labels and the their relationship is specified by:

(--rules rulestring,window)

NB: the rulestring and window term must be single quoted to prevent decapitation from the shell. Searching with rules is one of tacg's most sophisticated features allowing you to search for arbitrarily complex arrangements of patterns in a window of DNA. The rulestring above can be as complicated as you want it to be. The basic unit of the rulestring is a rule, and the basic unit of a rule is the construct Label:min:max. This means the pattern identified by Label must occur at least min times and not more than max times in the window for the rule to be passed. These basic units can be combined with the logicals AND (&), OR (|), and XOR (^, meaning A OR B, but not both), in whatever parenthetical arrangements you wish so it’s easy to make up quite complicated rules:

(Spacing in the rulestring doesn’t matter). The above patterns (ABCDE) have to be predefined in a rebase-style file and are restricted to one type per rule - you can’t combine a regex, IUPAC pattern and a matrix for example. Using this vocabulary, you can require that a pattern must be found (A:1:3), might be found (C:0:1), must be found a specific number of times (B:2:2 - B must be found 2 times), or must be found a variable number of times (B:7:9 - B must be found between 7-9 times). In one rulestring, a pattern can be required at one set of frequencies in one subrule (B:2:2) and at another in another subrule (B:7:9).

The window term is the length of the sliding window in which the rule is evaluated. Since the results will give you partial answers, I suggest you start large and decrease the window as necessary. The rule-based calculations are applied after all possible patterns are found, so if you are searching a long sequence such as a chromosome, you should reduce the patterns to those of interest. The actual rule calculations post-search are comparatively trivial.

Like the IUPAC and regex patterns entered from the commandline, the rule strings are appended to the tacg.patterns file in the current directory in a form amenable to being pasted into the rulefile (see below).

(--rulefile /path/to/rulefile)

As well, you can store multiple rules in a file (an example is rules.data, distributed with tacg) and search them all at once with this flag. The simple format is explained in the file.

tacg has several output formats ranging from a summary of hits to a complete linear map including full double stranded sequence, all hits, and co-translation in all frames. Almost all output is preceded by a notification that a subselection has been made (if it has been), the name of the input filename (if entered via commandline via STDIN, this is noted as fake filename; if entered via the web interface, it will be noted correctly), and the sequence composition, including degeneracies.

Here are the details, in roughly the order of popularity:

(-s)

This flag simply finds all patterns in the rebase file and summarizes the numeric results in a table. It is influenced by the RE selection criteria, so that as you make the criteria more selective, you will get fewer total patterns reported.

If you don’t specify specific patterns via the -p or -x options, tacg will go searching for the pattern database called rebase.data in the current directory, then in your HOME directory, then in the TACGLIB directory, before giving up. If you give it a restricted number of patterns, then -s will only show them. Also, if you filter the number of patterns with selection filters such as -o, -n, -cost, etc, the results will only show the ones that pass ALL the filters.

(-S {1*|2})

Produces a table of Site information, ordered by the appearance of the patterns in the rebase file or by the order that they have been entered from the command line if using the -p flag. If the -c flag is used, the Site entries will be ordered by number of hits, and then alphabetically.

(remainder omitted)

(-F {0*-3})

Produces tables of fragments much like the sites example above, with the fragments ordered by occurrence (-F1), size (-F2), or both (-F3). This flag is also affected by the -c flag, as above.

The output ordered by occurrence is:

The output ordered by size is:

(-L)

Minimally, produces a complete linear map of the sequence with all bases included. The patterns included are those that pass the selection filters by min, max, cost, overhang, size of recognition sequence, dam, and dcm methylation. The map can be modified in many ways, both to include extra information by requesting co-translations and less information to create a more compact map (see box below).

The following is the basic output format: generated by

Following is a stanza from a subsequence. The top strand is numbered startign at 1, The bottom strand is numbered starting at the original numbering (the output was generated with the command:

(-T {0*|1|3|6,1|3})

This option provides co-translation beneath the DNA sequence in 1,3, or 6 frames in 1 or 3 letter codes. It provides co-translation without any implication about ORFs, simply translating the nucleic acid triplets into amino acids. It uses the requested Codon Translation table for the translation. See also the -O flag and the --orfmap flag for additional Translation features.

(--numstart #)

Allows for starting the numbering for the linear map with something other than 1, for example to start numbering a genomic fragment containing a transcriptional unit to be numbered relative to the start site, you might use --numstart=-241 as below. Note that if you use the --numstart option, the lower strand will be numbered with the original numbering so that you can correlate the two (unless you also specify --strands=1 --notics, which will leave only the top strand alternatively numbered.

(--notics)

Reduces the output by one line per linear map stanza by removing the 5 and 10 base markers normally below the sequence; when combined with the --strands=1 (below) reduces it by 2, important for some.

(--strands {0*|1})

Reduces the output by one line per linear map stanza by removing the complementary strand; can be combined or used independently with the --notics option above.

(-l)

Requests a ladder map, much like GCG’s MAPPLOT output. When marking REs, the plot marks 5' overhangs with a ‘\, a 3’ overhang with `/ and blunt ends with a |. Patterns entered from the commandline are marked with a |. The Ladder map can be widened with the -w flag to increase resolution and the patterns can be Strider-sorted with the -c flag to collect patterns that hit the same number of times.

The Ladder Map ends with a Summary Map which plots infrequent patterns on a single line, using the same width and markers as the Ladder Map. The cutoff for the number of sites plotted is sequence-length dependent - the longer the sequence, the greater the cutoff.

shown without -c sorting option

shown with the -c sorting option

(-g {LoCutOff(,HiCutOff)})

Produces a crude simulation of an electrophoretic gel separation of the digest performed. It uses a simple log() approximation of the distance travelled. The ‘LoCutOff` sets the lower bound of the fragment size you want; the HiCutOff if entered, sets the upper bound - convenient if you want to compare different digestions sets or zoom into a particular region to get more resolution. The output (see Example output) uses patterns labels on the vertical axis and presents the fragment mobilities as moving from the right to the left. Single bands are indicated as a |. If there are more than one band that maps to a character position, that position is marked with the number of bands that the character position contains. More than 9 bands are marked with a *. As noted above, if the region in which you’re interested has multiple bands, you can 'zoom\’ in by specifying a smaller bracketing LoCutOff & HiCutOff, and also by expanding the width of the output. As in other restriction analyses, the output reflects the filtering by min/max # of hits , magnitude of site, overhang, cost, etc. the output can be further Strider-sorted with the -c flag.

(--clone {_,x…})

The --clone flag allows you to search for sequence segments amenable for cloning. You can specify segments where a cut MUST occur (#x#) and segments where they must NOT occur (#_#), where the # is a number in the range of the sequence length. You can chain up to 15 of these criteria together. The regions that satisfy the criteria either completely or partially will be printed in order of best-fit. You can additionally filter the REs used to attempt the cloning fit with the usual filtering agents: overhang, min, max, cost, etc.

This is a hard output to describe, but below is the output for the following command:

(-O {1-6(x),MinSize})

This flag will search any combination of the 6 frames for ORFs and report the ones greater than the MinSize in a table that includes:

The specification for the frames is simple but odd. All the frame numbers you want analysed are caoncatenated together. For example, if you wanted to analyze only frames 1 & 2 for ORFs larger than 150 AAs, the option string would be -O 12,150. For frames 1,3,4,6, the option string would be -O 1346,150.

The output format is described here but note that especially with degenerate sequences, the filtering is quite lax; degenerate nucleic acid sequence will simply cause unknown amino acids to be translated, and the resulting ORFs can be quite large, albeit weird (see above for Frame 3)

If you want extended information on each ORF, append an x to the frame specification (-o 1346x,150) and 3 more lines will be generated for each ORF entry listing the number and percentage of each Amino Acid.

Note that the actual translation of the DNA depends on the Codon table selected (see -C flag). Despite using the appropriate codon table, the translation may not be accurate because some organisms use non-MET initiators and tacg has not been coded to act on that information; it only starts ORFs on METs.(read more about this in the codon.data file in the Data directory)

NB: tacg is not bright enough to know that translations that cross the begin/end transition should be continued if the sequence is circular, so it will miss the ORFs that do this. If you are concerned about this, split your sequence in the middle and stick the begin/ends together and re-run the analysis. This also applies to the ORF maps, immediately below.

(--orfmap) When used with the -o flag above, this option generates a pseudographical ORF map showing the location and orientation of each ORF found, as well as a map of all starts and stops in the requested frames. Note that the ORF Map Summary will only show those ORFs that meet the length criteria.

(--ps), (--pdf)

These two options will generate a circular plasmid map (again, a la DNA Strider) in Postscript format (--ps) or PDF format (--pdf). The PDF format is the one that almost everyone will want; the postscript version is generated 1st and then converted to PDF so it’s identical. The conversion does require a working ghostscript installed as (or linked to) /usr/bin/gs, altho the creation of the map does not.

The map is appended to the file tacg_Map.ps in the current directory, and further runs will be appended to the same file, so it may be disconcerting to view the file at first; the later plots are at the end. If this behavior is not acceptable, prepend the tacg command with an rm tacg_Map.ps;

The sequence going into the plasmid map will be converted to a circular map regardless of whether the sequence is circular or not.

When the plasmid map option is called, the subroutine also calls the logdegen() function, which notes the start and end of every run of degeneracies. This allows the map to have the degeneracies marked along the rim as a grey band. This is particularly useful because it allows you to compose plasmid maps of known and unknown sequence and therefore get at least known sites marked correctly. Using the Gel Map option you can verify if the known sites generate valid fragment sizes.

(-C {0*-16})

The following translation tables are supported by tacg. The actual table is a slight modification of NCBI’s comprehensive Codon table collection. In that file (and in codon.data file, you can read the complete description of each.

Note that tacg does not provide as accurate translation as other tools. Specifically, it does not take into account the alternative initiator codons that some other species use, nor does it translate through the start/stop sequence in a file if you use the -f 0 flag to indicate a circular topology. These are known bugs and are already on the TODO list.

The following flags can be used to re-sort the order of various tables and maps, as well as expanding the width, and reformatting the output into one line output for easier chaining to other tools.

(-c)

So named for Christian Mark’s elegant DNA Strider for the Macintosh (a major influence and inspiration for tacg). This flag sorts the output for the Sites and Fragments tables and the Gel and Ladder Maps into a format which presents the results first by the number of hits and then alphabetically by label. You can easily see the results of this kind of sort by requesting output with sorting and without sorting.

(-w {1|#})

Enables you to set the ouput wider to gain resolution or reduce length. The numeric option can be either 1 which sets the output into a single line if possible for easier downstream analyses by other tools or to an integer between 60 (default) and 210. Numbers which are not exactly divisible by 15 are truncated to the next lower number which is. The numbers refer to the approximate width of the core result and the actual output is about 20 characters wider.

(-G {bin_size,X|Y|L})

This flag allows you to export the hit info as columnar data in 3 formats. The Y format (labels on top, data below), in particular can be used directly as gnuplot input. Each pattern gets its own column so you can plot the distribution with gnuplot using the following commands: In the following set of commands the line preceded by # are comments, not to be entered into the gnuplot interpreter:

The above stanza is printed in the output header as a reminder if you select the Y output format.

The extended format for the rebase file is described in the file itself. Minimally, it is identical to the GCG-formatted file from NEB. It can be extended to include extra information about the number of errors allowed, the cost, and dam/dcm sensitivity.

Like Strider, tacg uses a pre-calculated hashtable lookup of the restriction enzyme recognition sites, so that only about half of the sequence is checked any further than the initial hash (which itself is generated via the very efficient shift/add DFA algorithm). Depending on the degeneracy of the input sequence, the kind of output you request and the I/O of the machine, the program processes DNA at this speed:

tacg uses dynamic memory allocation for most data structures so that while there are a few hard-coded limitations (mostly in output format), it easily handles sequences into the 10s of Megabases.

As a practical rule of thumb, the program needs about 2 MB of free memory for itself, and then, depending on what results you’ve asked for, as much as 10 times the input sequence length to hold all the intermediate results before they are barfed to stdout. For the E coli genome (4.7Mb), tacg memory usage tops out at about 52 MB on a 32 bit Linux laptop.

In the Web form, there is a cutoff of 500,000 characters (not bases) as input; currently this includes extraneous characters such as headers, numbers, spaces, etc, so if you think you are being unfairly denied service, check how many bytes your file actually is. The command-line version is limited only by your machine’s RAM and your patience.

The included Web interface is a primitive (but usable, some say) attempt at wrapping the guts of tacg in a GUI. It’s a little hard to install (described above) but it you have a web server running and you understand how CGIs work, it should be a snap to get running. Or you can use one of the public tacg servers now running.

Nedit is well-designed, actively supported, free GUI editor for Xwindows. Besides its own features, NEdit allows you to seamlessly pipe selected text to external programs (…like tacg) and display the output in a NEdit window, and even add menu items of external programs (…like tacg), effectively making the NEdit/tacg combination a decent GUI biosequence analysis tool. I’ve included a few of my own shell commands and other NEdit settings in the dotnedit file accompanying the distribution. Either copy your nedit.rc to a safe place and start NEdit with mine, or edit the tasty bits of mine into yours.

If you’re just browsing, here’s my nedit.rc file. And incidentally, here’s another nedit.rc file from Thomas Siegmund which allows biosequence coloration inside Nedit.

While the Nedit/tacg combo doesn’t allow quite the same flexibility as Strider, it does allow you to use an exceptional editor to modify or annotate your sequence in a very natural way and also allows you to submit parts of that sequence for some of the usual analyses.