Forcing stdout and stderr to be written when a tool fails

CWL Questions
#1

When I am using cwltool to execute a CWL tool and an error occurs, nothing is written to standard output or standard error. I have been able to work around this by commenting out the offending portion of the CWL document and rerunning the tool to debug. Is there a better way to debug? Is there a way to force stdout and stderr to be written even when an error occurs?

Thanks!

#2

Do you mean the stdout or stderr of the tool? That should part of the logging already, there is nothing suppressing the logs. Unless they are being redirected by the tool definition, in which case it should still record output even if the tool failed.

#3

Thanks for your reply. Below is a simple tool that I created. I deliberately added a bug in the add.py script (sys.argv[4] rather than sys.argv[3]).

cwlVersion: v1.2
class: CommandLineTool
label: Adds two numbers
doc: |-
  This tool adds two integers and saves the sum to a file.
requirements:
  ShellCommandRequirement: {}
  InlineJavascriptRequirement: {}
  DockerRequirement:
    dockerImageId: add_tool
    dockerFile: |-
      FROM python:3.9-slim-buster
  NetworkAccess:
    class: NetworkAccess
    networkAccess: true
  InitialWorkDirRequirement:
    listing:
    - entryname: add.py
      entry: |-
        import sys
        
        number1 = int(sys.argv[1])
        number2 = int(sys.argv[2])
        out_file_path = sys.argv[4]
        
        with open(out_file_path, 'w') as out_file:
            total = number1 + number2
            out_file.write(str(total))
inputs:
  number1:
    type: int
    doc: |-
      An integer
  number2:
    type: int
    doc: |-
      A second integer
  output_file:
    type: string
    doc: |-
      Name of the output file that will be created.
      #Output_File=edam:format_1964
arguments:
  - shellQuote: false
    valueFrom: |-
      # This could be done with a bash command, but we're using Python for consistency with the other tool.

      python add.py $(inputs.number1) $(inputs.number2) $(inputs.output_file)
outputs:
  output_file:
    type: File
    outputBinding:
      glob: "$(inputs.output_file)"
    doc: |-
      Output file matching the name specified in the "output_file" input.
    format: edam:format_1964
  standard_output:
    type: stdout
    format: edam:format_1964
  standard_error:
    type: stderr
    format: edam:format_1964
stdout: stdout.txt
stderr: stderr.txt
 
s:author:
  - class: s:Person
    s:name: Stephen Piccolo
    s:identifier: https://orcid.org/0000-0003-2001-5640
 
s:dateCreated: "2020-07-14"
s:license: https://spdx.org/licenses/Apache-2.0
 
$namespaces:
  s: https://schema.org/
  edam: http://edamontology.org/
$schemas:
 - https://schema.org/version/latest/schemaorg-current-http.rdf
 - http://edamontology.org/EDAM_1.23.owl

Here is my input-object file:

number1: 99
number2: -83
output_file: result.txt

When I run the tool using cwltool, I get the following output:

**INFO** **[job add_tool.cwl] Max memory used: 0MiB**
**ERROR** **[job add_tool.cwl] Job error:**
**("Error collecting output for parameter 'output_file':\nadd_tool.cwl:53:7: Did not find output file with glob pattern: '['result.txt']'", {})**
**WARNING** **[job add_tool.cwl] completed permanentFail**
**{}**
**WARNING** **Final process status is permanentFail**

When this happens, it does not generate stdout.txt or stderr.txt or result.txt. So I cannot see the error message from my Python script unless I comment out the “output_file” output.

Any thoughts? There must be a better way for me to approach this.

Thanks again!

#4

Thanks @srp33 , I understand your question better now.

Can you open a GitHub issue at https://github.com/common-workflow-language/cwltool/issues/new requesting that cwltool “Preserve stdout/stderr outputs in case of permanentFail”

#5

@mrc Done, thank you.

#6

Hi Stephen,

I think what you really want is a cwltool feature that logs the stdout/stderr of each step, so that you don’t have to capture the stdout and stderr in the CWL itself. Right now you can capture the stderr of the entire cwltool run (cwltool 2> stderr.txt) but of course that mixes all the logging together which may be hard to read.

#7

When creating this ticket, I was focusing on a tool rather than a workflow. So at a minimum, it would be great if any stdout/stderr files that have been specified are created even on a permanentFail condition. In my example, my Python script throws an error, but I don’t see that error when a permanentFail occurs, even in cwltool’s outputs. So to debug it, I have to remove the cause of the permanentFail (in this case, the fact that the output files were not created) to see the error thrown by my Python script.

But of course, I could be missing something obvious.