Nextflow

Last Updated: December xx, 2025

Overview

You can run worflow jobs on compute nodes using Nextflow, which is developed as open-source software.

Note that the functionality to use the Technical Computing Suite (TCS) from Nextflow has been developed by volunteers, so it may be incomplete. If you encounter issues, please contact our center.

Preparation

To use Nextflow, you need to install it yourself. The basic steps are as follows:

1. Install Java
2. Download the nextflow program
3. Add it to your PATH (place the downloaded Nextflow program in a directory included in the PATH environment variable, or add a new PATH setting)

1. Install Java

$ curl -s https://get.sdkman.io | bash
# Necessary settings will be appended to the end of ~/.bashrc.
# Reopen the terminal (or log in again) to enable the sdk command.
$ sdk install java 17.0.10-tem
# If installed correctly, the java command will be available.

Example confirmation after installation:

[ku40000105@genkai0002 ~]$ sdk current java

Using java version 17.0.10-tem
[ku40000105@genkai0002 ~]$ java --version
openjdk 17.0.10 2024-01-16
OpenJDK Runtime Environment Temurin-17.0.10+7 (build 17.0.10+7)
OpenJDK 64-Bit Server VM Temurin-17.0.10+7 (build 17.0.10+7, mixed mode, sharing)
[ku40000105@genkai0002 ~]$

2. Download the nextflow program

As introduced on the official website, you can obtain the nextflow file in your current directory by executing a single command.

[ku40000105@genkai0002 tmp]$ curl -s https://get.nextflow.io | bash

      N E X T F L O W
      version 25.10.0 build 10289
      created 22-10-2025 16:26 UTC (23-10-2025 01:26 JDT)
      cite doi:10.1038/nbt.3820
      http://nextflow.io


Nextflow installation completed. Please note:
- the executable file `nextflow` has been created in the folder: /home/pj24001603/ku40000105/work/nextflow/tmp
- you may complete the installation by moving it to a directory in your $PATH

[ku40000105@genkai0002 tmp]$ ./nextflow -v
nextflow version 25.10.0.10289
[ku40000105@genkai0002 tmp]$

3. Place the executable in a directory included in your PATH

You can run it directly, but copying it as nextflow to a PATH-included directory is convenient. For example, on Genkai, ~/bin/ is included in the PATH by default, but the directory may not exist. Create it and copy the file there to make it accessible from anywhere.

[ku40000105@genkai0002 nextflow]$ mkdir ~/bin
[ku40000105@genkai0002 nextflow]$ cp ./nextflow ~/bin/
[ku40005108@genkai0002 nextflow]$ nextflow -v
nextflow version 25.10.0.10289
[ku40005108@genkai0002 nextflow]$

Operation Check 1

Verify basic functionality of Nextflow.

Running nextflow run hello will execute four “Hello world” jobs locally. These jobs are not submitted to compute nodes but run immediately.

[ku40005108@genkai0002 ~]$ nextflow run hello

 N E X T F L O W   ~  version 25.07.0-edge

Pulling nextflow-io/hello ...
 downloaded from https://github.com/nextflow-io/hello.git
Launching `https://github.com/nextflow-io/hello` [festering_wozniak] DSL2 - revision: 2ce0b0e294 [master]

executor >  local (4)
[13/6c7ddb] sayHello (1) [100%] 4 of 4 ✔
Ciao world!

Hello world!

Hola world!

Bonjour world!


[ku40005108@genkai0002 ~]$

To run jobs on compute nodes, a configuration file is required. For example, using the following nextflow.config file will execute each “Hello world” job on compute nodes in node group A. Resource information used at runtime is specified in clusterOptions.

[ku40005108@genkai0002 ~]$ cat nextflow.config
process {
  executor = 'tcs'
  time = '00:03:00'
  clusterOptions = '-L rscgrp=a-batch -L vnode-core=1'
}

The runtime output is as follows. You can monitor the job's execution status using tools like pjstat2.

Example output during job submission:

[ku40005108@genkai0002 ~]$ nextflow run hello

 N E X T F L O W   ~  version 25.10.0-edge

Launching `https://github.com/nextflow-io/hello` [big_visvesvaraya] DSL2 - revision: afff16a9b4 [master]

executor >  tcs (4)
[c8/935fd1] process > sayHello (3) [0%] 0 of 4
WARN: [INFO] PJM 0000 pjsub Job 4347504 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347505 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347506 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347507 submitted.

Example output after job completion:

[ku40005108@genkai0002 ~]$ nextflow run hello

 N E X T F L O W   ~  version 25.10.0

Launching `https://github.com/nextflow-io/hello` [big_visvesvaraya] DSL2 - revision: afff16a9b4 [master]

executor >  tcs (4)
[30/33565f] process > sayHello (2) [100%] 4 of 4 ✔
Hola world!

Hello world!

Bonjour world!

Ciao world!

WARN: [INFO] PJM 0000 pjsub Job 4347504 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347505 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347506 submitted.
WARN: [INFO] PJM 0000 pjsub Job 4347507 submitted.

Operation Check 2

Verify functionality using a sample program from Nextflow.

Many samples are stored in the tests directory of the repository. Here, we run the Basic pipeline introduced on the official website.

This sample performs the following steps:

1. Split the contents of the input file (sample.fa) into multiple files using the awk command (first job)
2. Reverse the contents of each file using the rev command (second job)
3. Display the output of step 2 to standard output

If you prepare the nextflow.config file mentioned earlier, the first and second jobs will be executed sequentially as batch jobs. Normally, TCS uses step jobs to control execution order, but with Nextflow, you can control job order without step jobs.

Example execution:

[ku40000105@genkai0001 tests]$ cat data/sample.fa
>1aboA
NLFVALYDFVASGDNTLSITKGEKLRVLGYNHNGEWCEAQTKNGQGWVPS
NYITPVN
>1ycsB
KGVIYALWDYEPQNDDELPMKEGDCMTIIHREDEDEIEWWWARLNDKEGY
VPRNLLGLYP
>1pht
GYQYRALYDYKKEREEDIDLHLGDILTVNKGSLVALGFSDGQEARPEEIG
WLNGYNETTGERGDFPGTYVEYIGRKKISP
>1vie
DRVRKKSGAAWQGQIVGWYCTNLTPEGYAVESEAHPGSVQIYPVAALERI
N
>1ihvA
NFRVYYRDSRDPVWKGPAKLLWKGEGAVVIQDNSDIKVVPRRKAKIIRD
[ku40000105@genkai0001 tests]$ nextflow basic-dsl2.nf

 N E X T F L O W   ~  version 25.10.0

Launching `basic-dsl2.nf` [pensive_becquerel] DSL2 - revision: 30435009f5

executor >  tcs (2)
[a6/b0ad64] splitSequences [100%] 1 of 1 ✔
[2d/eed513] reverse        [100%] 1 of 1 ✔
Aoba1>
SPVWGQGNKTQAECWEGNHNYGLVRLKEGKTISLTNDGSAVFDYLAVFLN
NVPTIYN
Bscy1>
YGEKDNLRAWWWEIEDEDERHIITMCDGEKMPLEDDNQPEYDWLAYIVGK
PYLGLLNRPV
thp1>
GIEEPRAEQGDSFGLAVLSGKNVTLIDGLHLDIDEEREKKYDYLARYQYG
PSIKKRGIYEVYTGPFDGREGTTENYGNLW
eiv1>
IRELAAVPYIQVSGPHAESEVAYGEPTLNTCYWGVIQGQWAAGSKKRVRD
N
Avhi1>
DRIIKAKRRPVVKIDSNDQIVVAGEGKWLLKAPGKWVPDRSDRYYVRFN

WARN: [INFO] PJM 0000 pjsub Job 4346348 submitted.


[ku40000105@genkai0001 tests]$ pjstat2 -H -l
 JOB_ID   JOB_NAME           MD  STATUS  USER        RSCGROUP     START_DATE           ELAPSE    NODE  CORE  GPU  POINT
 4346347  nf-splitSequences  NM  EXT     ku40000105  a-batch-low  2025/11/27 18:16:09  00:00:04     -     1    -      1
 4346348  nf-reverse         NM  EXT     ku40000105  a-batch-low  2025/11/27 18:16:33  00:00:01     -     1    -      1
# By examining the START_DATE and ELAPSE columns, you can confirm that the two jobs were executed sequentially.