Conditional in workflow error

CWL Questions
1

Hi! I’m trying to implement a conditional in a workflow using a boolean.

The workflow:

#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: Workflow
requirements:
   - class: StepInputExpressionRequirement
   - class: InlineJavascriptRequirement
   - class: MultipleInputFeatureRequirement

label: bowtie2 read mapping workflow

inputs:
  rna_data:
    type: boolean
    label: Treat as RNA data
    default: true

  prefix_id:
    type: string
    doc: prefix of the filename outputs
    default: output
  forward_reads:
    type: File
    format: http://edamontology.org/format_1931 # FASTQ
    doc: forward sequence file locally
    label: forward reads
  reverse_reads:
    type: File
    format: http://edamontology.org/format_1931 # FASTQ
    doc: reverse sequence file locally
    label: reverse reads
  indexfolder:
    type: Directory
    label: location of the indexes

  gtf:
    type: File
    format: http://edamontology.org/format_2306 # GTF
    doc: gtf file

steps:
  bowtie2:
    in:
      prefix: prefix_id
      forward_reads: forward_reads
      reverse_reads: reverse_reads
      indexfolder: indexfolder
    run: ../bowtie2/bowtie2_align_simple.cwl
    out:     
      [sam, metricsfile,bowtie2_stats]

  sam_to_sorted-bam:
    in:
      prefix: prefix_id
      sam: bowtie2/sam
    run: ../samtools/sam_to_sorted-bam.cwl
    out: 
      [sortedbam]

  featurecounts:
    in:
      prefix: prefix_id
      bam: sam_to_sorted-bam/sortedbam
      gtf: gtf
    when: $(inputs.rna_data)
    run: ../RNAseq/featurecounts.cwl
    out: 
      [readcounts]

outputs:
  bowtie2_bam:
    type: File
    outputSource: [sam_to_sorted-bam/sortedbam]
  readcounts:
    type: File
    outputSource: [featurecounts/readcounts]

$namespaces:
  s: https://schema.org/
  edam: http://edamontology.org/

$schemas:
 - http://edamontology.org/EDAM_1.18.owl

However this results in the error:

FO /usr/local/bin/cwltool 3.0.20200807132242
INFO [cwltool] /usr/local/bin/cwltool --no-container --preserve-entire-environment --outdir WORKFLOW --provenance WORKFLOW/PROVENANCE cwls/workflows/workflow_condtest.cwl /yaml/ctest.yaml
INFO Resolved 'cwls/workflows/workflow_condtest.cwl' to 'file:///cwls/workflows/workflow_condtest.cwl'
WARNING Workflow checker warning:
cwls/workflows/workflow_condtest.cwl:68:8: Source 'readcounts' of type "File" may be incompatible
cwls/workflows/workflow_condtest.cwl:75:5:   with sink 'readcounts' of type "File"
                                             Source is from conditional step, but pickValue is not
                                             used
INFO [provenance] Adding to RO file:///Unprocessed//F46_CTR_KO_T48_D1_1k_1.fq.gz
INFO [provenance] Adding to RO file:///Unprocessed//F46_CTR_KO_T48_D1_1k_2.fq.gz
INFO [provenance] Adding to RO file:///unlock/GenomeIndexes/GCA_900248155.1/GCA_900248155.1.ttl.gtf
INFO [workflow ] start
INFO [workflow ] starting step bowtie2
INFO [step bowtie2] start
INFO [job bowtie2] /tmp/tabn2y30$ /unlock/Binaries/bowtie2-2.4.1-linux-x86_64/bowtie2 \
    -S \
    output_bowtie2.sam \
    --met-file \
    output_bowtie2_metrics.txt \
    -1 \
    /tmp/tmpy6zfm1ok/stg1c40ebeb-bd67-4616-aa27-2e7c6a0f461b/F46_CTR_KO_T48_D1_1k_1.fq.gz \
    -x \
    /tmp/tmpy6zfm1ok/stg8be3a43a-263f-4f2d-ba40-b09b2fcb3bf1/bowtie2/GCA_900248155.1.ttl \
    -2 \
    /tmp/tmpy6zfm1ok/stg5ec7896b-11ea-4b9c-8674-c3fcfe7bae29/F46_CTR_KO_T48_D1_1k_2.fq.gz \
    --threads \
    1 2> /tmp/tabn2y30/output_bowtie2_stats.txt
INFO [job bowtie2] completed success
INFO [step bowtie2] completed success
INFO [workflow ] starting step sam_to_sorted-bam
INFO [step sam_to_sorted-bam] start
INFO [job sam_to_sorted-bam] /tmp/c8wg8yml$ /bin/sh \
    -c \
    /unlock/Binaries/samtools-1.10/samtools view -@ 1 -hu file:///tmp/tabn2y30/output_bowtie2.sam | /unlock/Binaries/samtools-1.10/samtools sort -@ 1 -o output.bam
INFO [job sam_to_sorted-bam] completed success
INFO [step sam_to_sorted-bam] completed success
INFO [workflow ] starting step featurecounts
ERROR Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "bam": {
04         "location": "file:///tmp/c8wg8yml/output.bam",
05         "basename": "output.bam",
06         "nameroot": "output",
07         "nameext": ".bam",
08         "class": "File",
09         "checksum": "sha1$e225df01ed1424b98dee520df24b91afd487bdf8",
10         "size": 131180,
11         "http://commonwl.org/cwltool#generation": 0,
12         "@id": "urn:uuid:b769694c-9e55-458c-93dd-cd7ca4cde09e"
13     },
14     "gtf": {
15         "class": "File",
16         "format": "http://edamontology.org/format_2306",
17         "location": "file:///unlock/GenomeIndexes/GCA_900248155.1/GCA_900248155.1.ttl.gtf",
18         "size": 6681561,
19         "basename": "GCA_900248155.1.ttl.gtf",
20         "nameroot": "GCA_900248155.1.ttl",
21         "nameext": ".gtf"
22     },
23     "prefix": "output"
24 };
25 var self = "output";
26 var runtime = {
27     "tmpdir": null,
28     "outdir": null
29 };
30 (function(){return ((inputs.rna_data));})()
stdout was: 'undefined'
stderr was: ''

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/cwltool/sandboxjs.py", line 411, in execjs
    return cast(CWLOutputType, json.loads(stdout))
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/cwltool/expression.py", line 397, in do_eval
    else 2,
  File "/usr/local/lib/python3.6/dist-packages/cwltool/expression.py", line 301, in interpolate
    js_console=js_console,
  File "/usr/local/lib/python3.6/dist-packages/cwltool/expression.py", line 256, in evaluator
    js_console=js_console,
  File "/usr/local/lib/python3.6/dist-packages/cwltool/sandboxjs.py", line 417, in execjs
    ) from err
cwltool.sandboxjs.JavascriptException: Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "bam": {
04         "location": "file:///tmp/c8wg8yml/output.bam",
05         "basename": "output.bam",
06         "nameroot": "output",
07         "nameext": ".bam",
08         "class": "File",
09         "checksum": "sha1$e225df01ed1424b98dee520df24b91afd487bdf8",
10         "size": 131180,
11         "http://commonwl.org/cwltool#generation": 0,
12         "@id": "urn:uuid:b769694c-9e55-458c-93dd-cd7ca4cde09e"
13     },
14     "gtf": {
15         "class": "File",
16         "format": "http://edamontology.org/format_2306",
17         "location": "file:///unlock/GenomeIndexes/GCA_900248155.1/GCA_900248155.1.ttl.gtf",
18         "size": 6681561,
19         "basename": "GCA_900248155.1.ttl.gtf",
20         "nameroot": "GCA_900248155.1.ttl",
21         "nameext": ".gtf"
22     },
23     "prefix": "output"
24 };
25 var self = "output";
26 var runtime = {
27     "tmpdir": null,
28     "outdir": null
29 };
30 (function(){return ((inputs.rna_data));})()
stdout was: 'undefined'
stderr was: ''

ERROR [step featurecounts] Cannot make job: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "bam": {
04         "location": "file:///tmp/c8wg8yml/output.bam",
05         "basename": "output.bam",
06         "nameroot": "output",
07         "nameext": ".bam",
08         "class": "File",
09         "checksum": "sha1$e225df01ed1424b98dee520df24b91afd487bdf8",
10         "size": 131180,
11         "http://commonwl.org/cwltool#generation": 0,
12         "@id": "urn:uuid:b769694c-9e55-458c-93dd-cd7ca4cde09e"
13     },
14     "gtf": {
15         "class": "File",
16         "format": "http://edamontology.org/format_2306",
17         "location": "file:///unlock/GenomeIndexes/GCA_900248155.1/GCA_900248155.1.ttl.gtf",
18         "size": 6681561,
19         "basename": "GCA_900248155.1.ttl.gtf",
20         "nameroot": "GCA_900248155.1.ttl",
21         "nameext": ".gtf"
22     },
23     "prefix": "output"
24 };
25 var self = "output";
26 var runtime = {
27     "tmpdir": null,
28     "outdir": null
29 };
30 (function(){return ((inputs.rna_data));})()
stdout was: 'undefined'
stderr was: ''

INFO [workflow ] completed permanentFail
{
    "bowtie2_bam": {
        "location": "file:///WORKFLOW/output.bam",
        "basename": "output.bam",
        "class": "File",
        "checksum": "sha1$e225df01ed1424b98dee520df24b91afd487bdf8",
        "size": 131180,
        "path": "/WORKFLOW/output.bam"
    },
    "readcounts": null
}
WARNING Final process status is permanentFail
INFO [provenance] Finalizing Research Object
INFO [provenance] Deleting existing /WORKFLOW/PROVENANCE
INFO [provenance] Research Object saved to /WORKFLOW/PROVENANCE

I looks like the conditional parameter has no value. When I change it to an integer of 1 for example and put:
when: $(inputs.rna_data == 1)
The step will be skipped.

I can’t see what I am doing wrong here.

Thanks in advance!

2

Should this not be true instead of == 1?

when: $(inputs.rna_data == true)

3

Should this not be true instead of == 1?

when: $(inputs.rna_data == true)

That will give no error but the step will be skipped, which it shouldn’t:
INFO [step featurecounts] will be skipped
INFO [step featurecounts] completed skipped

4

Hello @bartn, and welcome!

CWL doesn’t have a global namespace. As shown in the error output, for a when conditional the $inputs object only contains those parameters in that step’s in section. So to use when: $(inputs.rna_data) you need to include rna_data: rna_data within the in section of the featurecounts step. It is okay that the featurecounts.cwl doesn’t have an input named rna_data, it would get ignored. If there was a conflicting input parameter in featurecounts.cwl (perhaps an optional input that you don’t want to trigger, or was not a Boolean) then you can adjust the addition to the step’s in section: conditional: rna_data and use when: $(inputs.conditional)

5

That’s it! And thanks for explanation @mrc

1 Like