A "Job Error" for bet implementation

Hello, :slight_smile:
I wanted to implement Brain Extraction (bet) with cwl but I have encountered with this error that I cannot do anything to solve it. I would appreciate if you can kindly help me with it. :slight_smile:

Bet commandline must be something like this: bet and it will have two outputs if one add the ‘-m’, too.

My code in bet.cwl is:

___________________________________________
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: bet

inputs:
  rao:
    type: File
  out:
    type: string
  flagmi:
    type: boolean
    inputBinding:
      prefix: -m
outputs:
  - id: bet_output_mask
    type: File
    format: edam:format_3989
  - id: bet_output_mask1
    type: File
    format: edam:format_3989

$namespaces:
  edam: http://edamontology.org/
$schemas:
  - http://edamontology.org/EDAM_1.18.owl
___________________________________________

And I have this file as bet-job.yml :

rao:
  class: File
  location: STRUCT.nii.gz
  format: http://edamontology.org/edam:format_3989
out: STRUCT_brain.nii.gz
flagmi: true
_________________

The answer I get is:

----------------------------
/miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved 'bet1.cwl' to 'file:///home/ssaneei/Documents/cwl/example/bet/bet1.cwl'
INFO [job bet1.cwl] /tmp/t1q57sld$ bet \
    -m

Usage:    bet <input> <output> [options]

Main bet2 options:
  -o          generate brain surface outline overlaid onto original image
  -m          generate binary brain mask
  -s          generate approximate skull image
  -n          don't generate segmented brain image output
  -f <f>      fractional intensity threshold (0->1); default=0.5; smaller values give larger brain outline estimates
  -g <g>      vertical gradient in fractional intensity threshold (-1->1); default=0; positive values give larger brain outline at bottom, smaller at top
  -r <r>      head radius (mm not voxels); initial surface sphere is set to half of this
  -c <x y z>  centre-of-gravity (voxels not mm) of initial mesh surface.
  -t          apply thresholding to segmented brain image and mask
  -e          generates brain surface as mesh in .vtk format

Variations on default bet2 functionality (mutually exclusive options):
  (default)   just run bet2
  -R          robust brain centre estimation (iterates BET several times)
  -S          eye & optic nerve cleanup (can be useful in SIENA - disables -o option)
  -B          bias field & neck cleanup (can be useful in SIENA)
  -Z          improve BET if FOV is very small in Z (by temporarily padding end slices)
  -F          apply to 4D FMRI data (uses -f 0.3 and dilates brain mask slightly)
  -A          run bet2 and then betsurf to get additional skull and scalp surfaces (includes registrations)
  -A2 <T2>    as with -A, when also feeding in non-brain-extracted T2 (includes registrations)

Miscellaneous options:
  -v          verbose (switch on diagnostic messages)
  -h          display this help, then exits
  -d          debug (don't delete temporary intermediate images)

ERROR [job bet1.cwl] Job error:
Error validating output record. * the `bet_output_mask` field is not valid because
  is not a dict
* the `bet_output_mask1` field is not valid because
  is not a dict
 in {
    "bet_output_mask": null,
    "bet_output_mask1": null
}
WARNING [job bet1.cwl] completed permanentFail
{}
WARNING Final process status is permanentFail
--------------------------------------------------------------------

You need inputBinding on the input and output files as well.

Thanks for your answer.

For positions 1 and 2 I had this error again:

miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved ‘bet3.cwl’ to ‘file:///home/ssaneei/Documents/cwl/example/bet/bet3.cwl’
INFO [job bet3.cwl] /tmp/5lwzgeg3$ bet
-m
/tmp/77fsjruk/stgb17e1d55-011c-4571-8e60-77e19f575de7/STRUCT.nii.gz
STRUCT_brain.nii.gz
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory
/usr/local/fsl/bin/bet failed during command:-m /tmp/77fsjruk/stgb17e1d55-011c-4571-8e60-77e19f575de7/STRUCT.nii.gz STRUCT_brain.nii.gz
ERROR [job bet3.cwl] Job error:
Error validating output record. * the bet_output_mask field is not valid because
is not a dict

  • the bet_output_mask1 field is not valid because
    is not a dict
    in {
    “bet_output_mask”: null,
    “bet_output_mask1”: null
    }
    WARNING [job bet3.cwl] completed permanentFail
    {}
    WARNING Final process status is permanentFail

but I set a position for flag too. and now it is like this:

/miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved ‘bet3.cwl’ to ‘file:///home/ssaneei/Documents/cwl/example/bet/bet3.cwl’
INFO [job bet3.cwl] /tmp/lg5v4gs2$ bet
/tmp/i5yx78gm/stga6e1db1a-a94c-4912-83a9-0fe9d6bc5467/STRUCT.nii.gz
STRUCT_brain.nii.gz
-m
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory
/usr/local/fsl/bin/bet failed during command:/tmp/i5yx78gm/stga6e1db1a-a94c-4912-83a9-0fe9d6bc5467/STRUCT.nii.gz STRUCT_brain.nii.gz -m
ERROR [job bet3.cwl] Job error:
Error validating output record. * the bet_output_mask field is not valid because
is not a dict

  • the bet_output_mask1 field is not valid because
    is not a dict
    in {
    “bet_output_mask”: null,
    “bet_output_mask1”: null
    }
    WARNING [job bet3.cwl] completed permanentFail
    {}
    WARNING Final process status is permanentFail

As you can see, now the order of the commandline is correct but still the same error.

For your output files, you need an outputBinding , see Returning Output Files – Common Workflow Language User Guide

Though that error message is unhelpful. Can you open an issue at Sign in to GitHub · GitHub about the unhelpful error messages? They should have given specific advice.

Thanks.

indicating glob as:
outputBinding:
glob: *.nii.gz

will cause:

ERROR Tool definition failed validation:
bet3.cwl:24:13: found undefined alias ‘.nii.gz’

as it is not an identified type one, I suppose.

Turning it to:

outputBinding:
  glob: '*.nii.gz'

will rise:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:24:7: Did not find output file with glob pattern: ‘[’*.nii.gz’]’”, {})

adding java format:
outputBinding:
glob: $(’*.nii.gz’)

will cause:

[job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:24:7: Expression evaluation error:\nbet3.cwl:24:7: Syntax error in parameter reference ‘(’*.nii.gz’)’. This could be due to using Javascript code without specifying InlineJavascriptRequirement.”, {})

defining requirements:

requirements:
InlineJavascriptRequirement: {}

will have:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:27:7: Did not find output file with glob pattern: ‘[’*.nii.gz’]’”, {})

Using it like:
outputBinding:
glob: STRUCT_brain.nii.gz (or as string)

will have:
ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:19:7: Did not find output file with glob pattern: ‘[‘STRUCT_brain.nii.gz’]’”, {})

and I even tried to touch the file beforehand but the same error.

Also with help of stdout like:

stdout: STRUCT_brain.nii.gz

inputs:
rao:
type: File
inputBinding: {position: 1}
out:
type: string
inputBinding: {position: 2}
outputs:

  • id: bet_output_mask
    type: stdout
    format: edam:format_3989

I got:
line 157: /bin/remove_ext: No such file or directory

in the last one file was created but not filled.

And also stdout could be used just for one output, if I got it correctly.

Do you have any idea about the part that I am using in a wrong way?

Thanks in advance.

I solved it right now.
First of all, the major problem was the command bet itself, though I have tested it in commandline and it had been performed without error, I don’t know why it had problems with FSLDIR and it mentioned it as
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory

So I opened the bet file as administrator and add these lines in it, at line 155:

#FSL
FSLDIR=/usr/share/fsl
. ${FSLDIR}/5.0/etc/fslconf/fsl.sh
PATH=${FSLDIR}/5.0/bin:${PATH}
export FSLDIR PATH

and that error vanished. :partying_face:

The last version of my .cwl file is this one in which as my outputs were of one format, I used File because I had error:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:26:5: Multiple matches for output item that is a single file.”, {})

the last version :slight_smile:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: bet

inputs:
rao:
type: File
inputBinding: {position: 1}
out:
type: string
inputBinding: {position: 2}
flagmi:
type: boolean
inputBinding:
prefix: -m
position: 3
outputs:

  • id: bet_output_mask
    type: File
    format: edam:format_3989
    outputBinding:
    glob: “*.nii.gz”

$namespaces:
edam: http://edamontology.org/
$schemas:
-http://edamontology.org/EDAM_1.18.owl

1 Like

I’m so glad that you’ve solved your issue!

FYI, here are some tips for posting code on Discourse ( I recommend the block code formating) Discourse Guide: Code Formatting - Meta - Stonehearth Discourse