Accelerated Nanopore Base Calling with GPU

The advances in molecular biology and next generation sequencing technology allows us to generate vast amounts of sequencing data. Subsequently bioinformatics data analysis is often become a computationally demanding and time intensive task. This problem has been a major challenge in quite some time. To sped up the process, several approach are available to implement and will be discussing it shortly.

GPU Acceleration

Graphics Processing Unit or GPU is piece of hardware valued greatly by gamers, miners, and data scientist a like. As the name implies GPU is used mainly to process computer graphic, ie to render an image onto a display. GPU achieved this by breaking computational problem into a massively distributed processes within its architecture. This intensive number crunching happens simultaneously in parallel, thus accelerating the process way faster than a regular CPU.

Nowadays GPU acceleration utilized in many fields outside gaming for to tackle intensive computational tasks such as blockchain and machine learning among others. In the fields of Bioinformatics GPU acceleration is not something new either, several software is able to taking advantage of GPU to accelerate genomic data analysis. Hardware acceleration became necessary to deliver bioinformatics results quickly especially in a time sensitive conditions in healthcare.

In this article we will discuss how we can harness GPU parallelism to accelerate base calling process of a FAST5 data from Oxford Nanopore Technology sequencing platform. This article demonstrate each step in Google Collab notebook with GPU enabled.

If you want to use GPU for base calling in a local machine, start from CUDA with docker section at the end of article.

Google Colaboratory

Google Colaboratory or Colab is a cloud computing service product from Google. Colab is used to write and execute arbitrary python code through the browser without much of a setup. Colab is a hosted Jupyter notebook environment run entirely in the cloud. It is especially useful for learning and experimenting with python programs or Linux environments. Anyone can utilize a quite generous computing resource free of charge (and you can subscribe for more).

Let’s jump right to it, go to https://colab.research.google.com/ and create a new notebook.

Lets check colab available GPU

from tensorflow.python.client import device_lib
device_lib.list_local_devices()

You notice that the available GPU hardware to use is a NVIDIA T4. Nice. Now we need to download CUDA toolkit. CUDA is a application programming interface (API) that allows software to use certain types of graphics processing units for general purpose processing. In our case we want our base calling software able to interact with T4 GPU.

sudo apt-get --yes install cuda-toolkit-11-0 cuda-toolkit-10-2

To ensure the CUDA installed correctly, check GPU Status with following command.

%%bash
cd /usr/local/cuda/samples/1_Utilities/deviceQuery
sudo make
./deviceQuery

Base Calling with Guppy

Fantastic. Now we need to download Guppy. Guppy is a neural network based basecaller that in addition to basecalling also performs filtering of low quality reads, clipping of Oxford Nanopore adapters and estimation of methylation probabilities per base.

%%bash
wget https://mirror.oxfordnanoportal.com/software/analysis/ont-guppy_${version}_linux64.tar.gz
tar -zxvf ont-guppy_${version}_linux64.tar.gz

Check guppy installation.

%%bash
`pwd`/ont-guppy/bin/guppy_basecaller -v

All of our software needed is already set and now we need to download an example FAST5 data to work with. Lets get some data from nanopore-wgs-consortium/NA12878 github repository. Notice that we downloaded a data from a S3 bucket.

%%bash
wget http://s3.amazonaws.com/nanopore-human-wgs/rna/Multi_Fast5/Bham_Run1_20171115_cDNA_Multi/Bham_Run1_20171115_cDNA_0.fast5
http://s3.amazonaws.com/nanopore-human-wgs/rna/Bham_Run1_20171115_cDNA/Bham_Run1_20171115_cDNA_10180/reads/GA30000/4bc3cf4ba4a58c18a40a60b88bd63beebc1d69bf_Bham_Run1_20171115_cDNA/fast5/skip/0/GBX0017_20171115__GA30000_mux_scan_Bham_Run1_20171115_cDNA_10180_read_0_ch_271_strand.fast5

Run basecalling with CPU

%%bash
## GPU
`pwd`/ont-guppy/bin/guppy_basecaller -c dna_r9.4.1_450bps_fast.cfg -i `pwd`/sample/ -s fastq -x 'auto' --recursive

ONT Guppy basecalling software version 5.0.11+2b6dbff
config file:        /content/ont-guppy/data/dna_r9.4.1_450bps_fast.cfg
model file:         /content/ont-guppy/data/template_r9.4.1_450bps_fast.jsn
input path:         /content/sample/
save path:          fastq
chunk size:         2000
chunks per runner:  160
minimum qscore:     8
records per file:   4000
num basecallers:    4
gpu device:         auto
kernel path:        
runners per device: 8
Found 1 fast5 files to process.
Init time: 750 ms

0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************
Caller time: 3808 ms, Samples called: 42902386, samples/s: 1.12664e+07
Finishing up any open output files.
Basecalling completed successfully.

Run base calling with GPU

%%bash
## CPU
`pwd`/ont-guppy/bin/guppy_basecaller -c dna_r9.4.1_450bps_fast.cfg -i `pwd`/sample/ -s fastq --recursive

ONT Guppy basecalling software version 5.0.11+2b6dbff
config file:        /content/ont-guppy/data/dna_r9.4.1_450bps_fast.cfg
model file:         /content/ont-guppy/data/template_r9.4.1_450bps_fast.jsn
input path:         /content/sample/
save path:          fastq
chunk size:         2000
chunks per runner:  160
minimum qscore:     8
records per file:   4000
num basecallers:    1
cpu mode:           ON
threads per caller: 4
Found 1 fast5 files to process.
Init time: 27 ms

0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************
Caller time: 581472 ms, Samples called: 42902386, samples/s: 73782.4
Finishing up any open output files.
Basecalling completed successfully.

CUDA+Guppy with Docker

If you want to use GPU for base calling in a local machine follow the instruction bellow. Make sure Docker is installed and pull a image with a CUDA and Guppy inside.

docker pull genomicpariscentre/guppy-gpu

Run the following command to execute base calling in a container with GPU enabled.

docker run --gpus all -it -v `pwd`:/data genomicpariscentre/guppy-gpu guppy_basecaller -c dna_r9.4.1_450bps_fast.cfg -i /data/sample/ -s fastq --recursive