Providing input to a step that expects record input fields

CWL Questions
1

I currently have a cwl file that uses records to define sets of mutually exclusive or mutually required input parameters. I am now trying to use this cwl as a step in a workflow, but I don’t seem to understand how the input records for the step should be formatted.

I get the following tool definition error when trying to use this pipeline:

cwltool pipeline.cwl comp_dart_ex1.yml
INFO /home/ubuntu/miniconda3/bin/cwltool 3.0.20210319143721
INFO Resolved 'pipeline.cwl' to 'file:///mnt/cwl/spatial-transcriptomics-pipeline/pipeline.cwl'
ERROR Tool definition failed validation:
pipeline.cwl:2:1:     Object `pipeline.cwl` is not valid because
                      tried `Workflow` but
pipeline.cwl:291:1:     the `steps` field is not valid because
                          tried array of <WorkflowStep> but
pipeline.cwl:308:3:         item is invalid because
pipeline.cwl:310:5:           the `in` field is not valid because
pipeline.cwl:325:7:             item is invalid because
pipeline.cwl:326:9:               * invalid field `aux_names`, expected one of: 'id', 'source',
                                  'linkMerge', 'loadContents', 'loadListing', 'label', 'default', 'valueFrom'
pipeline.cwl:327:9:               * invalid field `aux_file_formats`, expected one of: 'id',
                                  'source', 'linkMerge', 'loadContents', 'loadListing', 'label', 'default',
                                  'valueFrom'
pipeline.cwl:328:9:               * invalid field `aux_file_vars`, expected one of: 'id', 'source',
                                  'linkMerge', 'loadContents', 'loadListing', 'label', 'default', 'valueFrom'
pipeline.cwl:329:9:               * invalid field `aux_cache_read_order`, expected one of: 'id',
                                  'source', 'linkMerge', 'loadContents', 'loadListing', 'label', 'default',
                                  'valueFrom'
pipeline.cwl:330:9:               * invalid field `aux_fixed_channel`, expected one of: 'id',
                                  'source', 'linkMerge', 'loadContents', 'loadListing', 'label', 'default',
                                  'valueFrom'

Notably, the four listed aux_* variables listed are a part of the same input record, as they are optional dependent parameters.

The most relevant thing I’ve found in the documentation for this is “Setting mutually exclusive parameters” under /user_guide/misc, but I don’t follow how this would work for records with multiple entries.

How should I define these input records (or is it bad practice to do this kind of variable specification on the individual steps, and they should only be specified as mutually independent/dependent at the workflow level)?

2

I found an answer to this, I couldn’t really find it in the documentation so I will document it here for posterity. The solution was to pass the entire record at the workflow level, the specifics of which would be parsed at the step level.

Basically changing this: (in the workflow cwl)

  in:
      aux_tilesets:
        aux_names: aux_names
        aux_file_formats: aux_file_formats
        aux_file_vars: aux_file_vars
        aux_cache_read_order: aux_cache_read_order
        aux_fixed_channel: aux_fixed_channel

To this:

 in:
      aux_tilesets: aux_tilesets
1 Like