Using output value of one workflow step as input in another

CWL Questions
1

Hello,

I’m trying to translate a bash script to a CWL workflow, however I’m having trouble connecting the two steps in the workflow.

This is the script I’m trying to translate to CWL. It gets the height of an image and divides it by 20. This value is then used as the size of a border that is added to the image.

#!/bin/bash

# Get the input file name
read -p "File name: " inputImage
# Base the border size on the image height
borderSize=$(($(identify -format "%h" $inputImage)/20))

# Get the border color
read -p "Color: " color
# Default color: Cyan
color=${color:-Cyan}
# Get output file name
read -p "Output filename: " outputImage
# Add the border
convert $inputImage -bordercolor $color -border $borderSize $outputImage

So far, I have made this CWL workflow (border.cwl):

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow

inputs:
  inputimage: File
  colorname: string
  outputimage: string
  
outputs:
  edited_image:
    type: File
    outputSource: add_border/border_image

steps:
  # The identify part of the shell script
  border_size:
    in:
      image: inputimage
    out: [bordersize]
    run:
      class: CommandLineTool
      baseCommand: [identify, -format]
      inputs:
        image:
          type: File
          inputBinding:
            position: 0
            prefix: "%h"
      outputs:
        bordersize:
          type: string
  # Add the border
  add_border:
    in:
      image: inputimage
      color: colorname
      size: border_size/bordersize
      outputname: outputimage
    out: [border_image]
    run:
      class: CommandLineTool
      baseCommand: [convert]
      inputs:
        image:
          type: File
          inputBinding:
            position: 0
        color:
          type: string
          default: Cyan
          inputBinding:
            position: 1
            prefix: -bordercolor
        size:
          type: string
          inputBinding:
            position: 2
            prefix: -border
        outputname:
          type: string
          inputBinding:
            position: 3
      outputs:
        border_image:
          type: File
          outputBinding:
            glob: $(inputs.outputname)

And this is the parameters file (border-params.cwl):

inputimage:
  class: File
  path: office.png
colorname: Cyan
outputimage: out.png

However, when I run the workflow, it does not work. This is part of the debug output when running cwltool --debug border.cwl border-params.cwl:

DEBUG [job border_size] initial work dir {}
INFO [job border_size] /tmp/e3qugpyo$ identify \
    -format \
    %h \
    /tmp/smk1wnsr/stg11a831ea-305e-42cc-9526-e4808c57ce58/office.png
1080DEBUG Could not collect memory usage, job ended before monitoring began.
ERROR [job border_size] Job error:
Error validating output record. the `bordersize` field is not valid because
  the value is not string
 in {
    "bordersize": null
}
WARNING [job border_size] completed permanentFail
DEBUG [job border_size] outputs {}
ERROR [step border_size] Output is missing expected field file:///.../CWL/workflowtest/border.cwl#border_size/bordersize
DEBUG [step border_size] produced output {}
WARNING [step border_size] completed permanentFail
DEBUG [job border_size] Removing input staging directory /tmp/smk1wnsr
DEBUG [job border_size] Removing temporary directory /tmp/g4j_fccv
INFO [workflow ] completed permanentFail

As you can see, the output of the first step is written to the console, but does not seem to be captured and used as the “size” input for the second workflow step.

I have tried using the “stdout” type for the “bordersize” output (as seen in the Common Workflow Language User Guide). However, this saves the output value to a file and I was unable to read the contents of that file for the input of the “add_border” step. It also feels unnecessarily complicated to save to a file first and then read it.

2

Hello!

You have to use the stdout and redirect it to a file with:

stdout: message

Then set the output to open it and use outputEval.

Something like:

outputs:
results:
type: string
outputBinding:
glob: message
loadContents: true
outputEval: $( self[0].contents.split("\n").join("") )

Hope this helps, typed on a phone…

1 Like
3

Hello FabriceBrito!

Thank you for your response. It solved the problem! The CWL workflow now does the exact same as the bash script.

I did have to add the InlineJavascriptRequirement, and I changed
self[0].contents.split("\n").join("") to $( self[0].contents ) as that is apparently enough (although it does also work with the split and join).

To give a clear overview, this is what border.cwl looks like now:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow

requirements:
  InlineJavascriptRequirement: {}

inputs:
  inputimage: File
  colorname: string
  outputimage: string
  
outputs:
  edited_image:
    type: File
    outputSource: add_border/border_image

steps:
  # The identify part of the shell script
  border_size:
    in:
      image: inputimage
    out: [bordersize]
    run:
      class: CommandLineTool
      baseCommand: [identify, -format]
      stdout: message
      inputs:
        image:
          type: File
          inputBinding:
            position: 0
            prefix: "%h"
      outputs:
        bordersize:
          type: int
          outputBinding:
            glob: message
            loadContents: true
            outputEval: $( self[0].contents / 20 )
  # Add the border
  add_border:
    in:
      image: inputimage
      color: colorname
      size: border_size/bordersize
      outputname: outputimage
    out: [border_image]
    run:
      class: CommandLineTool
      baseCommand: [convert]
      inputs:
        image:
          type: File
          inputBinding:
            position: 0
        color:
          type: string
          default: Cyan
          inputBinding:
            position: 1
            prefix: -bordercolor
        size:
          type: int
          inputBinding:
            position: 2
            prefix: -border
        outputname:
          type: string
          inputBinding:
            position: 3
      outputs:
        border_image:
          type: File
          outputBinding:
            glob: $(inputs.outputname)
2 Likes