How to run a bash command within a tool?

CWL Questions
1

I have a tool that utilizes a singularity container, however, I am on an HPC where the singularity module must be loaded prior to using a container. The command for this is “module load singularity”.

I’m trying to load the singularity module before running my command but the below code gives me an “Expression evaluation error”.

What am I doing wrong? Is there a better way to do this?

class: CommandLineTool
cwlVersion: v1.2

requirements:
  - class: InlineJavascriptRequirement

baseCommand: [ "bash", "-c" ]

inputs:
    in_gff:
        type: File

    singularity_image:
        type: string

arguments:  
    - valueFrom: |
        module load singularity && 
        singularity exec ${inputs.singularity_image} flatfile-to-json.pl --gff ${inputs.in_gff} --tracklabel 1 --out data/

outputs: 
    out_trackList:
        type: File
        outputBinding: 
            glob: data/trackList.json
    out_tracks:
        type: Directory
        outputBinding: 
            glob: data/tracks
2

To use JavaScript in your valueFrom, you have to specify whether you want to return a single value ($()) or run JavaScript code that will return a value (${}). You can read more about it here.

For your example, check if this produces the desired result:

arguments:  
    - valueFrom: |
        ${
          return `module load singularity && 
          singularity exec ${inputs.singularity_image} flatfile-to-json.pl --gff ${inputs.in_gff} --tracklabel 1 --out data/`
        }

The idea is to write some code that will be wrapped in a JavaScript code like (function() { .... })(), so it must return a string that will be appended to bash -c. I think the backticks are supported, and they allow for multiline strings. Otherwise, you can just return a simple JS string.

3

It worked after I added quotes, otherwise bash tried to execute “module”.

arguments:  
    - valueFrom: |
        ${
        return `"module load singularityCE && 
        singularity exec ${inputs.singularity_image} flatfile-to-json.pl --gff ${inputs.in_gff} --tracklabel 1 --out json"`
        }
4

InitialWorkDirRequirement might make this a little more readable

requirements:
  InitialWorkDirRequirement:
    listing:
      - entryname: run.sh
        entry: |
        #!/usr/bin/env bash
        
        # Set up
        set -euo pipefail
        
        # Load singularity
        module load singularity
        
        # Run Singularity exec
        singularity exec "$(inputs.singluarity_image.path)" \\
          flatfile-to-json.pl \\
            --gff "$(inputs.in_gff.path)" \\
            --tracklabel 1 \\
            --out data/

baseCommand: [ 'bash', 'run.sh' ]

Loading singularity into the tool doesn’t seem like the right way to go about it, and certainly reduces the reproducibility.

Have you tried using something like

requirements:
  DockerPull: docker/uri/to/jbrowse

baseCommand: [ "flatfile-to-json.pl" ]

...

And running cwltool with the --singlularity parameter?

1 Like