A "Job Error" for bet implementation

sRah_saneE · August 5, 2021, 11:56am

Hello,
I wanted to implement Brain Extraction (bet) with cwl but I have encountered with this error that I cannot do anything to solve it. I would appreciate if you can kindly help me with it.

Bet commandline must be something like this: bet and it will have two outputs if one add the ‘-m’, too.

My code in bet.cwl is:

___________________________________________
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: bet

inputs:
  rao:
    type: File
  out:
    type: string
  flagmi:
    type: boolean
    inputBinding:
      prefix: -m
outputs:
  - id: bet_output_mask
    type: File
    format: edam:format_3989
  - id: bet_output_mask1
    type: File
    format: edam:format_3989

$namespaces:
  edam: http://edamontology.org/
$schemas:
  - http://edamontology.org/EDAM_1.18.owl
___________________________________________

And I have this file as bet-job.yml :

rao:
  class: File
  location: STRUCT.nii.gz
  format: http://edamontology.org/edam:format_3989
out: STRUCT_brain.nii.gz
flagmi: true
_________________

The answer I get is:

----------------------------
/miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved 'bet1.cwl' to 'file:///home/ssaneei/Documents/cwl/example/bet/bet1.cwl'
INFO [job bet1.cwl] /tmp/t1q57sld$ bet \
    -m

Usage:    bet <input> <output> [options]

Main bet2 options:
  -o          generate brain surface outline overlaid onto original image
  -m          generate binary brain mask
  -s          generate approximate skull image
  -n          don't generate segmented brain image output
  -f <f>      fractional intensity threshold (0->1); default=0.5; smaller values give larger brain outline estimates
  -g <g>      vertical gradient in fractional intensity threshold (-1->1); default=0; positive values give larger brain outline at bottom, smaller at top
  -r <r>      head radius (mm not voxels); initial surface sphere is set to half of this
  -c <x y z>  centre-of-gravity (voxels not mm) of initial mesh surface.
  -t          apply thresholding to segmented brain image and mask
  -e          generates brain surface as mesh in .vtk format

Variations on default bet2 functionality (mutually exclusive options):
  (default)   just run bet2
  -R          robust brain centre estimation (iterates BET several times)
  -S          eye & optic nerve cleanup (can be useful in SIENA - disables -o option)
  -B          bias field & neck cleanup (can be useful in SIENA)
  -Z          improve BET if FOV is very small in Z (by temporarily padding end slices)
  -F          apply to 4D FMRI data (uses -f 0.3 and dilates brain mask slightly)
  -A          run bet2 and then betsurf to get additional skull and scalp surfaces (includes registrations)
  -A2 <T2>    as with -A, when also feeding in non-brain-extracted T2 (includes registrations)

Miscellaneous options:
  -v          verbose (switch on diagnostic messages)
  -h          display this help, then exits
  -d          debug (don't delete temporary intermediate images)

ERROR [job bet1.cwl] Job error:
Error validating output record. * the `bet_output_mask` field is not valid because
  is not a dict
* the `bet_output_mask1` field is not valid because
  is not a dict
 in {
    "bet_output_mask": null,
    "bet_output_mask1": null
}
WARNING [job bet1.cwl] completed permanentFail
{}
WARNING Final process status is permanentFail
--------------------------------------------------------------------

tetron · August 5, 2021, 12:58pm

You need inputBinding on the input and output files as well.

sRah_saneE:

  rao:
    type: File
    inputBinding: {position: 1}
  out:
    type: string
    type: File
    inputBinding: {position: 2}
  flagmi:
    type: boolean
    inputBinding:
      prefix: -m

sRah_saneE · August 5, 2021, 1:08pm

Thanks for your answer.

For positions 1 and 2 I had this error again:

miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved ‘bet3.cwl’ to ‘file:///home/ssaneei/Documents/cwl/example/bet/bet3.cwl’
INFO [job bet3.cwl] /tmp/5lwzgeg3$ bet
-m
/tmp/77fsjruk/stgb17e1d55-011c-4571-8e60-77e19f575de7/STRUCT.nii.gz
STRUCT_brain.nii.gz
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory
/usr/local/fsl/bin/bet failed during command:-m /tmp/77fsjruk/stgb17e1d55-011c-4571-8e60-77e19f575de7/STRUCT.nii.gz STRUCT_brain.nii.gz
ERROR [job bet3.cwl] Job error:
Error validating output record. * the bet_output_mask field is not valid because
is not a dict

the bet_output_mask1 field is not valid because
is not a dict
in {
“bet_output_mask”: null,
“bet_output_mask1”: null
}
WARNING [job bet3.cwl] completed permanentFail
{}
WARNING Final process status is permanentFail

but I set a position for flag too. and now it is like this:

/miniconda3/envs/onto38/bin/cwl-runner 3.1.20210628163208
INFO Resolved ‘bet3.cwl’ to ‘file:///home/ssaneei/Documents/cwl/example/bet/bet3.cwl’
INFO [job bet3.cwl] /tmp/lg5v4gs2$ bet
/tmp/i5yx78gm/stga6e1db1a-a94c-4912-83a9-0fe9d6bc5467/STRUCT.nii.gz
STRUCT_brain.nii.gz
-m
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory
/usr/local/fsl/bin/bet failed during command:/tmp/i5yx78gm/stga6e1db1a-a94c-4912-83a9-0fe9d6bc5467/STRUCT.nii.gz STRUCT_brain.nii.gz -m
ERROR [job bet3.cwl] Job error:
Error validating output record. * the bet_output_mask field is not valid because
is not a dict

the bet_output_mask1 field is not valid because
is not a dict
in {
“bet_output_mask”: null,
“bet_output_mask1”: null
}
WARNING [job bet3.cwl] completed permanentFail
{}
WARNING Final process status is permanentFail

As you can see, now the order of the commandline is correct but still the same error.

mrc · August 5, 2021, 6:21pm

For your output files, you need an outputBinding , see Returning Output Files – Common Workflow Language User Guide

Though that error message is unhelpful. Can you open an issue at Sign in to GitHub · GitHub about the unhelpful error messages? They should have given specific advice.

sRah_saneE · August 6, 2021, 7:48am

Thanks.

indicating glob as:
outputBinding:
glob: *.nii.gz

will cause:

ERROR Tool definition failed validation:
bet3.cwl:24:13: found undefined alias ‘.nii.gz’

as it is not an identified type one, I suppose.

Turning it to:

outputBinding:
  glob: '*.nii.gz'

will rise:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:24:7: Did not find output file with glob pattern: ‘[’*.nii.gz’]’”, {})

adding java format:
outputBinding:
glob: $(’*.nii.gz’)

will cause:

[job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:24:7: Expression evaluation error:\nbet3.cwl:24:7: Syntax error in parameter reference ‘(’*.nii.gz’)’. This could be due to using Javascript code without specifying InlineJavascriptRequirement.”, {})

defining requirements:

requirements:
InlineJavascriptRequirement: {}

will have:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:27:7: Did not find output file with glob pattern: ‘[’*.nii.gz’]’”, {})

Using it like:
outputBinding:
glob: STRUCT_brain.nii.gz (or as string)

will have:
ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:19:7: Did not find output file with glob pattern: ‘[‘STRUCT_brain.nii.gz’]’”, {})

and I even tried to touch the file beforehand but the same error.

Also with help of stdout like:

stdout: STRUCT_brain.nii.gz

inputs:
rao:
type: File
inputBinding: {position: 1}
out:
type: string
inputBinding: {position: 2}
outputs:

id: bet_output_mask
type: stdout
format: edam:format_3989

I got:
line 157: /bin/remove_ext: No such file or directory

in the last one file was created but not filled.

And also stdout could be used just for one output, if I got it correctly.

Do you have any idea about the part that I am using in a wrong way?

Thanks in advance.

sRah_saneE · August 6, 2021, 1:37pm

I solved it right now.
First of all, the major problem was the command bet itself, though I have tested it in commandline and it had been performed without error, I don’t know why it had problems with FSLDIR and it mentioned it as
/usr/local/fsl/bin/bet: line 157: /bin/remove_ext: No such file or directory

So I opened the bet file as administrator and add these lines in it, at line 155:

#FSL
FSLDIR=/usr/share/fsl
. ${FSLDIR}/5.0/etc/fslconf/fsl.sh
PATH=${FSLDIR}/5.0/bin:${PATH}
export FSLDIR PATH

and that error vanished.

The last version of my .cwl file is this one in which as my outputs were of one format, I used File because I had error:

ERROR [job bet3.cwl] Job error:
(“Error collecting output for parameter ‘bet_output_mask’:\nbet3.cwl:26:5: Multiple matches for output item that is a single file.”, {})

the last version

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: bet

inputs:
rao:
type: File
inputBinding: {position: 1}
out:
type: string
inputBinding: {position: 2}
flagmi:
type: boolean
inputBinding:
prefix: -m
position: 3
outputs:

id: bet_output_mask
type: File
format: edam:format_3989
outputBinding:
glob: “*.nii.gz”

$namespaces:
edam: http://edamontology.org/
$schemas:
-http://edamontology.org/EDAM_1.18.owl

mrc · August 25, 2021, 12:24pm

I’m so glad that you’ve solved your issue!

FYI, here are some tips for posting code on Discourse ( I recommend the block code formating) Discourse Guide: Code Formatting - Meta - Stonehearth Discourse

Topic		Replies	Views
Beginner's question: stdin from generated file CWL Questions	2	417	June 4, 2021
Potential defect in ontology format checking CWL Questions	6	472	March 1, 2022
HaplotypeCallerSpark CWL Questions	4	467	November 11, 2020
CWL workflow using ncbi sra tool kit fails CWL Questions	12	458	October 6, 2023
Conformance tests are failing for a certain CWL runner CWL Questions	8	352	November 2, 2022

A "Job Error" for bet implementation

the last version

Related Topics