Skip to main content

Crate broccoli_rs

Crate broccoli_rs 

Source
Expand description

Broccoli

Broccoli is designed to infer orthologous groups and orthologous pairs using a mixed phylogeny-network approach. It also detects chimeric proteins resulting from gene-fusion events and assigns these proteins to the corresponding orthologous groups.

This page is the user manual of the broccoli command-line program.

Overview of the 4 steps of Broccoli

§Installation

The easiest way to install Broccoli is via Bioconda, which also installs DIAMOND and FastTree:

conda install bioconda::broccoli

Alternatively, with a Rust toolchain installed, it can be built from the GitHub repository:

git clone https://github.com/rderelle/Broccoli.git
cd Broccoli
cargo build --release     # executable: target/release/broccoli

Broccoli uses two external programs during step 2 (both are installed by Bioconda):

  • DIAMOND v0.9.30 or above (similarity searches). The -combined_search option needs a DIAMOND release that supports --taxon-k.
  • FastTree v2.1.11 or above, in its single-threaded version (phylogenetic analyses; Broccoli runs many FastTree instances in parallel). FastTree is not needed with -phylogenies nj.

By default they should be named diamond and fasttree and be in your PATH; otherwise, give their location with -path_diamond and -path_fasttree.

§Quick start

The only mandatory argument is the directory containing the proteome files:

# display the help menu
broccoli -help

# run the whole pipeline on 8 threads
broccoli -dir proteomes -threads 8

# fastest settings: one combined DIAMOND search and built-in NJ trees
broccoli -dir proteomes -threads 8 -combined_search -phylogenies nj

# orthologous groups only (steps 1 to 3), with maximum-likelihood trees
broccoli -dir proteomes -threads 8 -steps 1,2,3 -phylogenies ml

# write the dir_step* directories in output_broccoli/ instead of the current directory
broccoli -dir proteomes -threads 8 -output output_broccoli

Broccoli writes its temporary and output files in 4 directories, dir_step1 to dir_step4 (one per step), created in the current directory, or in the directory given with -output (e.g. -output output_broccoli). The main results are:

  • dir_step3/orthologous_groups.txt
  • dir_step3/chimeric_proteins.txt
  • dir_step4/orthologous_pairs.txt

§Input data

The input is a directory containing one proteome file per species:

  • sequences in FASTA format;
  • file extension .fas, .fasta or .faa (any case), optionally gzipped (e.g. human.faa.gz); other files in the directory are ignored;
  • the protein name is the part of the FASTA header before the first space. Protein names should be unique across all files: if not, Broccoli prints a warning and lists the duplicates in dir_step1/duplicate_names.txt.

Species are identified by their file names in all output tables, so naming the files after the species (e.g. Homo_sapiens.fasta) makes the results easier to read.

§How Broccoli works

The analysis is divided into 4 steps, each writing its files in its own directory:

  1. k-mer simplification (dir_step1): within each proteome, proteins sharing a k-mer (i.e. an identical fragment) are grouped and only the longest protein of each group is kept for the next steps. This removes redundancy (isoforms, identical copies) and speeds up the analysis. The removed proteins are added back, alongside their representative, in the final outputs.
  2. phylomes (dir_step2): all-against-all similarity searches with DIAMOND (keeping the best hits in each species), then, for each protein whose hits contain several proteins of the same species, a phylogenetic tree is built from the trimmed alignment of its hits.
  3. network analysis (dir_step3): orthologous relationships are extracted from the trees and combined into an orthology network, in which orthologous groups are identified by label propagation. Chimeric proteins are then detected and assigned to all their orthologous groups, and spurious hits are removed.
  4. orthologous pairs (dir_step4): within each orthologous group, pairs of proteins are classified as orthologs or paralogs from their relationships in all the trees they appear in.

Two things to keep in mind:

  • each step requires the directories generated by the previous steps, so steps can be run separately (e.g. -steps 1,2,3 then later -steps 4), but always from the same directory, or with the same -output;
  • each step first deletes its own directory if it exists, to avoid mixing files from different runs. Rename or move the results you want to keep before re-running a step, or use a different -output for each analysis.

§Output files

Orthologous groups are named OG_1, OG_2, … and contain proteins from at least 2 species. Every step also writes a log file (log_step1.txt to log_step3.txt) in its directory.

§Orthologous groups and chimeric proteins (dir_step3)

FileContent
orthologous_groups.txtone orthologous group per line: its name, then the names of its proteins separated by spaces
table_OGs_protein_counts.txtnumber of proteins of each species (columns) in each orthologous group (rows)
table_OGs_protein_names.txtsame table with the protein names instead of their number
chimeric_proteins.txtchimeric proteins: species file, protein name, number and list of the orthologous groups it belongs to
unclassified_proteins.txtproteins not assigned to any orthologous group
statistics_per_OG.txtfor each orthologous group: number of species, number of proteins after the k-mer simplification and in total, clustering coefficient
statistics_per_species.txtpercentage and number of proteins of each species assigned to an orthologous group
statistics_nb_OGs_VS_nb_species.txtnumber of orthologous groups containing 2, 3, … species
log_step3.txtsize of the orthology network, distribution of the edge weights, numbers of communities and chimeric proteins

OGs_in_network.txt is an intermediate file used by step 4.

§Orthologous pairs (dir_step4)

orthologous_pairs.txt lists one pair of orthologous proteins per line (two protein names separated by a tab).

§Other files

  • dir_step1/log_step1.txt: number of proteins in each proteome before and after the k-mer simplification.
  • dir_step2/log_step2.txt: for each species, number of proteins with a phylogeny, without a phylogeny (no duplicated species among their hits), with an empty alignment after trimming, and with a failed tree.
  • the other files in dir_step1 and dir_step2 (reduced proteomes, similarity hits, trees) are intermediate files used by the next steps.

§Options

Options are given with a single dash (e.g. -threads 8); -help (or -h) prints the list of options with their default values.

§General options

OptionDefaultDescription
-steps1,2,3,4steps to perform, comma separated, consecutive
-threads1number of threads
-output, -ocurrent directorydirectory where the dir_step* directories are written (created if needed)

-steps selects which steps are performed. For instance, if you only need orthologous groups, use -steps 1,2,3. The steps must be consecutive (-steps 1,3 is not valid).

§Step 1: k-mer simplification

OptionDefaultDescription
-dirrequireddirectory containing the proteome files (full or relative path)
-kmer_size100length of the k-mers
-kmer_min_aa15minimum number of different amino acids a k-mer should contain

Proteins of the same proteome sharing at least one k-mer of length -kmer_size are grouped, and only the longest one is kept. K-mers made of fewer than -kmer_min_aa different amino acids, or containing X or *, are ignored to avoid grouping proteins through low-complexity or repetitive regions.

§Step 2: phylomes

OptionDefaultDescription
-path_diamonddiamondpath of the DIAMOND executable
-path_fasttreefasttreepath of the FastTree executable
-e_value0.001e-value threshold of the similarity searches
-nb_hits6maximum number of hits per species
-max_gap0.7maximum fraction of gaps per alignment position
-phylogeniesbionjtree-building method: bionj, me, ml or nj
-combined_searchoffone DIAMOND search against a combined database

-e_value and -nb_hits correspond to the DIAMOND options --evalue and --max-target-seqs (applied per species). -max_gap controls the trimming of the alignments: positions with a fraction of gaps equal to or above this value are removed.

-phylogenies sets how gene trees are built:

  • bionj: FastTree BioNJ (fast);
  • me: FastTree minimum evolution (slower, more precise);
  • ml: FastTree maximum likelihood (much slower, most precise);
  • nj: built-in neighbor-joining, much faster per tree than FastTree and does not require FastTree; results are only marginally less accurate than with bionj.

By default, Broccoli runs one DIAMOND search per pair of proteomes. With -combined_search, it runs a single search of all proteins against one database containing all proteomes, which avoids repeating the database and query setup and makes step 2 much faster, with near-identical hits. Two caveats: it uses more memory, and since e-values are computed with the mean proteome size, it should not be used with proteomes of very different sizes.

§Step 3: network analysis

OptionDefaultDescription
-sp_overlap0.5maximum ratio of overlapping species in phylogenetic trees
-min_weight0.1minimum weight for an edge to be kept in the orthology network
-min_nb_hits2spurious hits: minimum number of hits within the orthologous group
-chimeric_shared0.5chimeric proteins: minimum fraction of connected proteins in each orthologous group
-chimeric_nb_sp3chimeric proteins: minimum number of species connected in each orthologous group

-sp_overlap controls how orthologous groups are delineated in the trees. At each node, the ratio is the maximum fraction of species shared between the leaves already browsed and the sister leaves. Low values (e.g. 0.3) produce small, tightly defined orthologous groups, while high values (e.g. 0.7) produce larger groups, similar to the orthogroups of OrthoFinder.

-min_weight cleans the orthology network by removing weak edges (the weight of an edge is relative to the strongest edge of each protein). Besides removing spurious connections, this speeds up step 3.

The 3 remaining options control the two corrections applied after the identification of orthologous groups:

  • spurious hits: a protein is removed from its orthologous group if it has fewer than -min_nb_hits DIAMOND hits with other proteins of the group (only applied to groups containing more than -min_nb_hits + 1 proteins);
  • chimeric proteins: a protein connected to several orthologous groups through non-overlapping regions of its sequence is considered chimeric (gene fusion) if, for each group, it is connected to at least a fraction -chimeric_shared of the group’s proteins, belonging to at least -chimeric_nb_sp species. It is then assigned to all these groups.

§Step 4: orthologous pairs

OptionDefaultDescription
-ratio_ortho0.5minimum ratio ortho / (ortho + para) to report a pair
-not_same_spoffignore orthologous relationships between proteins of the same species

For each pair of proteins of an orthologous group, Broccoli counts how many times they appear as orthologs or as paralogs across the trees. -ratio_ortho sets the precision/recall trade-off: increasing it improves the precision of the orthologous pairs, decreasing it improves their recall.

-not_same_sp discards pairs of proteins from the same species (e.g. for the Quest for Orthologs benchmark).

§Tips and FAQ

§How to make Broccoli faster?

The most time-consuming parts of the analysis are the similarity searches and the phylogenetic analyses (step 2):

  • -combined_search speeds up the similarity searches (see the caveats in step 2);
  • -phylogenies nj speeds up the phylogenetic analyses;
  • reducing -nb_hits reduces the number of sequences in alignments and trees, at a potential cost in precision;
  • reducing -kmer_size simplifies the proteomes further at step 1, at the risk of grouping paralogs together. This has nearly no effect on prokaryotic proteomes, and in some species many unrelated proteins share identical fragments (e.g. viral domains in Trichomonas vaginalis).

As an example, with 60 fungal proteomes on 8 threads (steps 1 to 3; Intel Xeon Platinum 8358 @ 2.60 GHz):

OptionsRuntime (min)Peak memory (GB)
default4194
-combined_search19618
-combined_search -phylogenies nj10618

§How to make Broccoli more precise?

Minimum-evolution or maximum-likelihood trees (-phylogenies me or -phylogenies ml) improve the delineation of closely related orthologous groups, but increase the runtime of the phylogenetic analyses by a factor of about 30 and 100 respectively.

Lowering -sp_overlap (e.g. 0.4) improves precision but lowers recall (small orthologous groups missing some divergent in-paralogs), which can be suitable for the identification of phylogenomic markers.