.cwl workflow find file in subdirectories of an input directory

CWL Questions
#1

I have output files from a previous step organized as files inside subdirectories of an output directory. Such as:

/home/enrico/Dropbox/NY/app/GATK_CNV_germline/scatterIntervals/scattered_hs37d5.preprocessed.300bp.primary_contigs.noBL.filtered.F10
├── temp_0001_of_37
│   └── scattered.interval_list
├── temp_0002_of_37
│   └── scattered.interval_list
├── temp_0003_of_37
│   └── scattered.interval_list

I need to pass the initial dir as input to a following step of the workflow and it to be able to find all files inside its subdirectories. I’ve tried this:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow
inputs:
  my_dir: Directory

outputs: []

requirements:
  StepInputExpressionRequirement: {}
  ScatterFeatureRequirement: {}

steps:
  zeroth_step:
    run:
      class: ExpressionTool
      requirements: { InlineJavascriptRequirement: {} }
      inputs:
        dir: Directory
      expression: '${return {"inner_directories": inputs.dir.listing};}'
      outputs:
        inner_directories: Directory[]
    in:
      dir: my_dir
    out: [inner_directories]

  first_step:
    run:
      class: ExpressionTool
      requirements: { InlineJavascriptRequirement: {} }
      inputs:
        my_dir:
          type: Directory
      expression: '${return {"files": inputs.my_dir.listing};}'
      outputs:
        files: File[]
    in:
      my_dir: zeroth_step/inner_directories
    scatter: my_dir
    out: [files]

  second_step:
    run:
      class: CommandLineTool
      inputs:
        my_file:
          type: File[]
      baseCommand: echo
      outputs: []
    in:
      my_file: first_step/files
    out: []

But I got this error:

Tool definition failed validation:
GermlineCNVCaller-scattered-workflow-0.cwl:41:11: Source ‘files’ of type {“items”: {“type”: “array”, “items”: “File”}, “type”: “array”} is incompatible
GermlineCNVCaller-scattered-workflow-0.cwl:52:7: with sink ‘my_file’ of type {“type”: “array”, “items”: “File”}

Cause it recognizes is as a nested array. How can I workaround it?

Thank you so much in advance for any help!

#2

Hi! I see you have a list of directories as output of your zeroth step, and as input a single directory for your first step. That is a type mismatch.

The error message text looks odd to me, and might be a cwltool bug. But this is on top of the port mismatch.