Apply basename to an array on input file

CWL Questions
1

I am trying to get the basenames for an array of input files in CWL, but get a Key Error when I try the code below. I cannot seem to find a way to have $(self.basename) be applied to an array of input files. It works fine with one input file.

This is the cwl runner code

cwlVersion: v1.0
class: CommandLineTool
inputs:
  filesB:
    type: File[]
    inputBinding:
      position: 2
      valueFrom: $(self.basename)
outputs:
    []
baseCommand: echo

This is the input yml file

   - class: File
     path: /aap/noot/mies/one.txt
   - class: File
     path: /tmp/two.txt

This is the result:

/usr/bin/cwl-runner 1.0.20180302231433
Resolved '/arrays.cwl' to 'file:///arrays.cwl'
[job arrays.cwl] initializing from file:///arrays.cwl
[job arrays.cwl] {
    "filesB": [
        {
            "class": "File",
            "location": "file:///aap/noot/mies/one.txt",
            "size": 14,
            "basename": "one.txt",
            "nameroot": "one",
            "nameext": ".txt"
        },
        {
            "class": "File",
            "location": "file:///tmp/two.txt",
            "size": 4,
            "basename": "two.txt",
            "nameroot": "two",
            "nameext": ".txt"
        }
    ]
}
Got workflow error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cwltool/executors.py", line 98, in run_jobs
    for r in jobiter:
  File "/usr/lib/python2.7/dist-packages/cwltool/command_line_tool.py", line 379, in job
    visit_class([builder.files, builder.bindings], ("File", "Directory"), _check_adjust)
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 51, in visit_class
    visit_class(d, cls, op)
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 51, in visit_class
    visit_class(d, cls, op)
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 48, in visit_class
    visit_class(rec[d], cls, op)
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 51, in visit_class
    visit_class(d, cls, op)
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 46, in visit_class
    op(rec)
  File "/usr/lib/python2.7/dist-packages/cwltool/command_line_tool.py", line 181, in check_adjust
    f["path"] = docker_windows_path_adjust(builder.pathmapper.mapper(f["location"])[1])
  File "/usr/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 290, in mapper
    return self._pathmap[src]
KeyError: 'file:///aap/noot/mies/one.txt'
Workflow error, try again with --debug for more information:
'file:///aap/noot/mies/one.txt'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cwltool/main.py", line 588, in main
    **vars(args))
  File "/usr/lib/python2.7/dist-packages/cwltool/executors.py", line 29, in __call__
    return self.execute(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cwltool/executors.py", line 69, in execute
    self.run_jobs(t, job_order_object, logger, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cwltool/executors.py", line 113, in run_jobs
    raise WorkflowException(Text(e))
WorkflowException: 'file:///aap/noot/mies/one.txt'

I was expecting this: (Edited to what I think it would look like)

/usr/bin/cwl-runner 1.0.20180302231433
Resolved '/arrays.cwl' to 'file:///arrays.cwl'
[job arrays.cwl] /tmp/tmpdB3bMX$ echo \
    one.txt \
    two.txt
one.txt two.txt
[job arrays.cwl] completed success
{}
Final process status is success

I am guessing I have to change the valueFrom field to apply the basename to an array, but cannot figure out how.

2

Hello @thondeboer and thanks for your question!

The trick is that for arrays we can specify inputBinding at two different levels

First lets use the expanded notation for an array of File objects:

inputs:
  filesB:
    type:
      type: array
      items: File
    inputBinding:
      position: 2
      valueFrom: $(self.basename)

At https://www.commonwl.org/user_guide/09-array-inputs/index.html we learn that

The inputBinding can appear either on the outer array parameter definition or the inner array element definition, and these produce different behavior when constructing the command line

So we can move the valueFrom into an inputBinding on the inner array element definition.

All together we get:

cwlVersion: v1.0
class: CommandLineTool
inputs:
  filesB:
    type:
      type: array
      items: File
      inputBinding:
        valueFrom: $(self.basename)
    inputBinding:
      position: 2
outputs:
    []
baseCommand: echo

$ cwltool stackoverflow_58532163.cwl --filesB README.rst --filesB CONTRIBUTING.md 
INFO /usr/bin/cwltool 1.0.20190915164430
INFO Resolved 'stackoverflow_58532163.cwl' to 'file:///home/michael/cwltool/stackoverflow_58532163.cwl'
INFO [job stackoverflow_58532163.cwl] /tmp/5u7ouspa$ echo \
    README.rst \
    CONTRIBUTING.md
README.rst CONTRIBUTING.md
INFO [job stackoverflow_58532163.cwl] completed success
{}
INFO Final process status is success
1 Like