Installation
phyluce uses a number of tools that allow it to assemble data, search for UCE loci, align resulting reads, manipulate alignments, prepare alignments for analysis, etc. To accomplish these goals, phyluce uses wrappers around a number of programs that do each of these tasks (sometimes phyluce can use several different programs that accomplish the same task in different ways). As a result, the dependency chain (the programs that phyluce requires to run) is reasonably complex.
In the current versions (> 1.7.x), we very strongly suggest that users install phyluce using the miniconda Python distribution.
Attention
We do not support installing phyluce through means other than the conda installer. This means that we do not test phyluce against any binaries, other than those we build and distribute through conda. Although you can configure phyluce to use binaries of different provenance, this is not officially supported.
Attention
We do not support phyluce on Windows, although you technically should be able to install phyluce on Windows using the Windows Subsystem for Linux (WSL) and installing Ubuntu 20.04 LTS from the Windows Store. You should also be able to use the docker image.
Note
We build and test the binaries available through conda using 64-bit operating systems that include:
MacOS 10.15
Ubuntu 20.04 LTS
We will officially support MacOS 10.16 when the github build system offers this platform for automated tests.
phyluce is also available for use as a docker image. Underneath the hood the docker image runs Ubuntu 20.04 LTS and installs phyluce and related packages using conda.
Install Process
Using Conda
The installation process is a 2-step process. You need to:
Installing phyluce will install all of the required binaries, libraries, and Python dependencies that you need to run the program.
Install miniconda
First, you need to install miniconda. Follow the instructions for your platform that are available from conda.io. After you have run the install process be sure that you:
close and re-open your terminal window
run
conda list
which should produce output
Install phyluce
Current practice with conda is to keep all environments separate and not
to use the base environment as a “default” environment. So, we will be
installing phyluce into an environment named phyluce-X.x.x
where the
X.x.x
represents the version you choose.
Go to the phyluce github release page
Download the appropriate
*.yml
file for the phyluce version you want and the operating system you are usingInstall that into an environment corresponding to the phyluce version, e.g.
phyluce-1.7.0
following the instructions on the phyluce release page
This will create an environment named phyluce-X.x.x
, then download and install
everything you need to run phyluce into this phyluce-X.x.x
conda environment.
To use your new phyluce environment, you must run (replace X.x.x
with the
correct version):
conda activate phyluce-X.x.x
To stop using this phyluce environment, you must run:
conda deactivate
What conda installs
When you install phyluce, it specifies a number of dependencies that it needs
to run. If you would like to know everything that conda has
installed, you can open up the *.yml
you downloaded (it is simply a text file)
and take a look at the contents.
From within the conda environment, you can also run
conda activate phyluce-X.x.x
conda list
Added benefits
An added benefit of using conda is that you can also run all of the 3rd-party binaries without worrying about setting the correct $PATH, etc.
For example, phyluce requires MUSCLE for installation, and MUSCLE was installed
by conda as a dependency of phyluce. Because conda puts all of these binaries
in your $PATH
when the environment is activateed, we can also just run MUSCLE
on the command-line, with, e.g.,:
$ muscle -version
MUSCLE v3.8.1551 by Robert C. Edgar
Using Docker
We also provide phyluce as a docker image, which means you can run the phyluce installation anywhere that you can run docker. The docker image is built on Ubuntu 20.04 LTS using conda. To pull the docker image:
Go to the phyluce github release page
Find the phyluce release you want (usually the most recent)
Run the
docker pull
command listed
Although using docker is beyond the scope of this guide, you can run phyluce within a docker using a command similar to the following, e.g.:
docker run fairclothlab/phyluce:<version> phyluce <phyluce_program_name>
Where <version>
corresponds to the version of phyluce you are using. When you run this, all commands are run in the default directory /work
and the user within the container is named phyluce
.
You will very likely want to mount a local directory (on your computer) to this /work
directory in the docker container and make yourself the owner of the result files. If you are working in /home/you/phyluce
on your computer, you can accomplish all of by running phyluce like (using phyluce 1.7.0 as an example):
docker run \
-v /home/you/phyluce:/data \
--user $(id -u):$(id -g) \
fairclothlab/phyluce:1.7.0 \
phyluce_assembly_assemblo_spades \
--output spades-test \
--config assembly.conf \
--cores 12
The -v /home/you/phyluce:/data
maps your directory (/home/you/phyluce
) onto the container working directory (/work
), the --user $(id -u):$(id -g)
makes the owner of the files in the container your user and group, the fairclothlab/phyluce:1.7.0
is the name of the image to use, and the rest are standard phyluce commands.
Finally, you may want to run many commands in the docker container (e.g. as in an entire analysis run). This can be accomplished by starting a bash_ shell in the container, and working from within the container’s bash prompt, as in:
docker run \
-v /home/you/phyluce:/data \
--user $(id -u):$(id -g) \
-i -t fairclothlab/phyluce:1.7.0 \
/bin/bash
# this drops you into the shell, where you can run commands, e.g.:
@d51aa2f2d565:/data$ phyluce_assembly_assemblo_spades -h
Using Singularity
If you are using Singularity, you should be able to pull the Docker image, and convert it for use, although this is not tested and is not supported. For example:
singularity pull docker://fairclothlab/phyluce:1.7.0
If that does not work, you could also use the phyluce Dockerfile to create a Singularity definition file, and build a Singularity image.
phyluce configuration
As of v1.5.x, phyluce uses a configuration file to keep track of paths to relevant
binaries, as well as some configuration information. This file is located at
$CONDA_PREFIX/phyluce/config
. Although you can edit this file directly, you
can also create a user-specific configuration file at ~/.phyluce.conf (note the
preceding dot), which will override the default values with different paths.
So, if you need to use a slightly different binary or you want to experiment with new binaries (e.g. for assembly), then you can change the paths in this file rather than deal with hard-coded paths.
Attention
This WILL NOT work for the docker image by default. You also do NOT need create this file unless you realy know what you are doing.
Warning
Changing the $PATHs in the config file can break things pretty
substantially, so please use with caution. If you are making changes,
edit the copy at ~/.phyluce.conf
) rather than the default copy.
The format of the config file as of v1.7.0 looks similar to the following:
[binaries]
abyss:$CONDA/bin/ABYSS
abyss-pe:$CONDA/bin/abyss-pe
bcftools:$CONDA/bin/bcftools
bedtools:$CONDA/bin/bedtools
bwa:$CONDA/bin/bwa
gblocks:$CONDA/bin/Gblocks
lastz:$CONDA/bin/lastz
mafft:$CONDA/bin/mafft
muscle:$CONDA/bin/muscle
pilon:$CONDA/bin/pilon
raxml-ng:$CONDA/bin/raxml-ng
samtools:$CONDA/bin/samtools
seqtk:$CONDA/bin/seqtk
spades:$CONDA/bin/spades.py
trimal:$CONDA/bin/trimal
velvetg:$CONDA/bin/velvetg
velveth:$CONDA/bin/velveth
snakemake:$CONDA/bin/Snakemake
[workflows]
mapping:$WORKFLOWS/mapping/Snakefile
correction:$WORKFLOWS/contig-correction/Snakefile
phasing:$WORKFLOWS/phasing/Snakefile
#----------------
# Advanced
#----------------
[headers]
trinity:comp\d+_c\d+_seq\d+|c\d+_g\d+_i\d+|TR\d+\|c\d+_g\d+_i\d+|TRINITY_DN\d+_c\d+_g\d+_i\d+
velvet:node_\d+
abyss:node_\d+
idba:contig-\d+_\d+
spades:NODE_\d+_length_\d+_cov_\d+.\d+
[spades]
max_memory:4
cov_cutoff:5
Other useful tools
You will need to be familiar with the command-line/terminal, and it helps to have a decent text editor for your platform. Here are some suggestions that are free: