mapPredicate error: subworkflow & scattering using dotproduct

CWL Questions
1

This is the first time attempting to use scattering features, as well as the subworkflow schema. Unsure how to resolve the follow error for my code:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow

requirements:
  ScatterFeatureRequirement: {}
  SubworkflowFeatureRequirement: {}

inputs:
  pdb_id_array: string[]
  pdb_filename_array: string[]
  pdbH_file_array: string[]
  CS_path_array: string[]
  CS_file_array: string[]
  temp_filename_array: string[]

steps:
  subworkflow:
    run:
      class: Workflow
      inputs:
        pdb_id: string
        pdb_filename: string
        chain: string
        py_buildPDB: File
        cfg_file: File
        temp_filename: string
        pdbH_file: string
        add_hydrogen_script: File
        CS_path: string
        CS_file: string
        shift_predict_script: File
        pH_value: float
        temp: float
        outputFormat: string
        cs_predictor: string
        task_insertCS: string
        task_buildPDB: string
      outputs: []
      steps:
        create_pdbFile:
          run: buildPDB.cwl
          in:
            pdbID: pdb_id
            pdbFilename: pdb_filename
            Chain: chain
            cfgFile: cfg_file
            pyBuildPDB: py_buildPDB
            taskBuildPDB: task_buildPDB
          out: [pdbFile]
        protonate:
        run: addHydrogen.cwl
        in:
          addHydrogen_script: add_hydrogen_script
          pdb_to_addHydrogens: create_pdbFile/pdbFile
          protonatedPDB: pdbH_file
          tempFile: temp_filename
        out: [pdbH_File]
      predictChemShift:
        run: shiftPredict.cwl
        in:
          shiftPredict_script: shift_predict_script
          protonated_pdb: protonate/pdbH_File
          chemical_shiftDir: CS_path
          chemical_shiftName: CS_file
          format: outputFormat
          pH: pH_value
          temperature: temp
        out: [CS_file]
      populateDatabase:
        run: populate_database.cwl
        in:
          pyInsertCS: py_buildPDB
          pH: pH_value
          temperature: temp
          csPredictor: cs_predictor
          chemicalShifts: predictChemShift/CS_file
          Chain: chain
          pdbID: pdb_id
          cfgFile: cfg_file
          taskInsertCS: task_insertCS
        out: []
    scatter:
    in:
      pdb_id: pdb_id_array
      pdb_filename: pdb_filename_array
      pdbH_file: pdbH_file_array
      cs_path: cs_path_array
      cs_file: cs_file_array
      temp_filename: temp_filename_array
    scatterMethod: dotproduct
    out: []
outputs:
  dictFile:
    type: File[]
    outputSource: [create_pdbFile/pdbFile, protonate/pdbH_File, predictChemShift/CS_file]

YML File

pdb_id: [2TRX, 1UB4]
pdb_filename: [2TRX.pdb, 1UB4.pdb]
chain: A
py_buildPDB:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/resources/buildPDBfile.py
cfg_file:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/configs/query.cfg
pdbH_file: [2TRX_protonated.pdb, 1UB4_protonated.pdb]
temp_filename: [/local/PycharmProjects/CSPworkflow/code/usage/temp.pdb, /local/PycharmProjects/CSPworkflow/code/usage/temp2.pdb]
add_hydrogen_script:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/addHydrogen.sh
CS_path: [/local/PycharmProjects/CSPworkflow/code/usage/2TRX_chemicalShift.csv, /local/PycharmProjects/CSPworkflow/code/usage/1UB4_chemicalShift.csv]
CS_file: [2TRX_chemicalShift2.csv, 1UB4_chemicalShift2.csv]
outputFormat: CSV
pH_value: 7.0
temp: 298.0
shift_predict_script:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/shiftPredict.sh
cs_predictor: shiftx2
task_buildPDB: build_pdb
task_insertCS: insert_cs

Error:

usage$ cwl-runner csp_shiftx.cwl csp_shiftx.yml 
/usr/bin/cwl-runner 1.0.20180302231433
Resolved 'csp_shiftx.cwl' to 'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl'
loader is 139650196365712
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 860, in resolve_all
    val, base_url, file_base=file_base, checklinks=False)
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 851, in resolve_all
    self._resolve_idmap(document, loader)
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 631, in _resolve_idmap
    "and does not have a mapPredicate", k, v)
ValidationException: ("mapSubject '%s' value '%s' is not a dictand does not have a mapPredicate", 'out', None)
failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 890, in resolve_all
    val, base_url, file_base=file_base, checklinks=False)
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 864, in resolve_all
    id(loader), file_base, key, validate.indent(six.text_type(v))))
ValidationException: (139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in field run:
  ("mapSubject '%s' value '%s' is not a dictand does not have a mapPredicate", 'out', None)
loader is 139650196365712
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 860, in resolve_all
    val, base_url, file_base=file_base, checklinks=False)
  File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver.py", line 895, in resolve_all
    id(loader), file_base, i, validate.indent(six.text_type(v))))
ValidationException: (139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in position 0:
  (139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in field run:
    ("mapSubject '%s' value '%s' is not a dictand does not have a mapPredicate", 'out', None)
Tool definition failed validation:
(139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in field steps:
  (139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in position 0:
    (139650196365712) (file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl) Validation error in field run:
      ("mapSubject '%s' value '%s' is not a dictand does not have a mapPredicate", 'out', None)
2

Modified the code and resulted in the following errors:

csp.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow

requirements:
  - class: ScatterFeatureRequirement
  - class: SubworkflowFeatureRequirement

inputs:
  pdb_id_array: string[]
  pdb_filename_array: string[]
  pdbH_file_array: string[]
  CS_path_array: string[]
  CS_file_array: string[]
  temp_file_array: string[]
  chain: string
  py_buildPDB: File
  cfg_file: File
  add_hydrogen_script: File
  shift_predict_script: File
  pH_value: float
  temp: float
  outputFormat: string
  cs_predictor: string
  task_insertCS: string
  task_buildPDB: string
outputs:
  output_files:
    type: File[]
    outputSource: subworkflow1/output_files

steps:
  subworkflow1:
    in:
      pdb_id: pdb_id_array
      pdb_filename: pdb_filename_array
      pdbH_file: pdbH_file_array
      CS_path: CS_path_array
      CS_file: CS_file_array
      temp_file: temp_file_array
    scatter: [pdb_id, pdb_filename, pdbH_file, CS_path, CS_file, temp_file]
    out: [output_files]
    run:
      class: Workflow
      requirements:
        - class: ScatterFeatureRequirement
      inputs:
        pdb_id: string
        pdb_filename: string
        chain: string
        py_buildPDB: File
        cfg_file: File
        temp_file: string
        pdbH_file: string
        add_hydrogen_script: File
        CS_path: string
        CS_file: string
        shift_predict_script: File
        pH_value: float
        temp: float
        outputFormat: string
        cs_predictor: string
        task_insertCS: string
        task_buildPDB: string
      outputs:
        output_files:
          type: File[]
          outputSource: [create_pdbFile/pdbFile, protonate/pdbH_File, predictChemShift/CS_file]
      steps:
        create_pdbFile:
          run: buildPDB.cwl
          scatter: [pdbID, pdbFilename]
          scatterMethod: dotproduct
          in:
            pdbID: pdb_id
            pdbFilename: pdb_filename
            Chain: chain
            cfgFile: cfg_file
            pyBuildPDB: py_buildPDB
            taskBuildPDB: task_buildPDB
          out: [pdbFile]
        protonate:
          run: addHydrogen.cwl
          scatter: [tempFile, protonatedPDB]
          scatterMethod: dotproduct
          in:
            addHydrogen_script: add_hydrogen_script
            pdb_to_addHydrogens: create_pdbFile/pdbFile
            protonatedPDB: pdbH_file
            tempFile: temp_file
          out: [pdbH_File]
        predictChemShift:
          run: shiftPredict.cwl
          scatter: [chemical_shiftDir, chemical_shiftName]
          scatterMethod: dotproduct
          in:
            shiftPredict_script: shift_predict_script
            protonated_pdb: protonate/pdbH_File
            chemical_shiftDir: CS_path
            chemical_shiftName: CS_file
            format: outputFormat
            pH: pH_value
            temperature: temp
          out: [CS_file]
        populateDatabase:
          run: populate_database.cwl
          scatter: [pdbID]
          in:
            pyInsertCS: py_buildPDB
            pH: pH_value
            temperature: temp
            csPredictor: cs_predictor
            chemicalShifts: predictChemShift/CS_file
            Chain: chain
            pdbID: pdb_id
            cfgFile: cfg_file
            taskInsertCS: task_insertCS
          out: []

csp.yml

pdb_id: [2TRX, 1UB4]
pdb_filename: [2TRX.pdb, 1UB4.pdb]
chain: A
py_buildPDB:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/resources/buildPDBfile.py
cfg_file:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/configs/query.cfg
pdbH_file: [2TRX_protonated.pdb, 1UB4_protonated.pdb]
temp_file: [/local/PycharmProjects/CSPworkflow/code/usage/temp.pdb, /local/PycharmProjects/CSPworkflow/code/usage/temp2.pdb]
add_hydrogen_script:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/addHydrogen.sh
CS_path: [/local/PycharmProjects/CSPworkflow/code/usage/2TRX_chemicalShift.csv, /local/PycharmProjects/CSPworkflow/code/usage/1UB4_chemicalShift.csv]
CS_file: [2TRX_chemicalShift2.csv, 1UB4_chemicalShift2.csv]
outputFormat: CSV
pH_value: 7.0
temp: 298.0
shift_predict_script:
  class: File
  path: /local/PycharmProjects/CSPworkflow/code/usage/shiftPredict.sh
cs_predictor: shiftx2
task_buildPDB: build_pdb
task_insertCS: insert_cs

buildPDB.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: python3

inputs:
  pyBuildPDB:
    type: File
    inputBinding:
      position: 0
  taskBuildPDB:
    type: string
    inputBinding:
      position: 1
      prefix: --task
  pdbID:
    type: string
    inputBinding:
      position: 2
      prefix: --pdb_id
  Chain:
    type: string
    inputBinding:
      position: 3
      prefix: --chain
  pdbFilename:
    type: string
  cfgFile:
    type: File
    inputBinding:
      position: 4
      prefix: --cfg_file

outputs:
  pdbFile:
    type: File
    outputBinding:
      glob: $(inputs.pdbFilename)

addHydrogen.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: sh

inputs:
  addHydrogen_script:
    type: File
    inputBinding:
      position: 0
  pdb_to_addHydrogens:
    type: File
    inputBinding:
      position: 1
  tempFile:
    type: string
    inputBinding:
      position: 2
  protonatedPDB:
    type: string
    inputBinding:
      position: 3

outputs:
  pdbH_File:
    type: File
    outputBinding:
      glob: $(inputs.protonatedPDB)

shiftPredict.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
#baseCommand: [python, /usr/software/SHIFTX2/shiftx2.py]
#baseCommand: [bash, /home/nmrbox/lcraft/Desktop/caricWorkflow_FM/shiftPredict.sh]
baseCommand: bash

inputs:
  shiftPredict_script:
    type: File
    inputBinding:
      position: 0
  protonated_pdb:
#    type: string
    type: File
    inputBinding:
      position: 1
#      prefix: -i
  chemical_shiftDir:
    type: string
    inputBinding:
      position: 2
#      prefix: -o
  temperature:
    type: float
    inputBinding:
      position: 3
#      prefix: -t
  pH:
    type: float
    inputBinding:
      position: 4
#      prefix: -p
  format:
    type: string
    inputBinding:
      position: 5
#      prefix: -f
  chemical_shiftName:
    type: string
    inputBinding:
      position: 6

#outputs: []
outputs:
  CS_file:
    type: File
    outputBinding:
      glob: $(inputs.chemical_shiftName)

populate_database.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: python3

inputs:
  pyInsertCS:
    type: File
    inputBinding:
      position: 0
  taskInsertCS:
    type: string
    inputBinding:
      position: 1
      prefix: --task
  pdbID:
    type: string
    inputBinding:
      position: 2
      prefix: --pdb_id
  Chain:
    type: string
    inputBinding:
      position: 3
      prefix: --chain
  cfgFile:
    type: File
    inputBinding:
      position: 4
      prefix: --cfg_file
  csPredictor:
    type: string
    inputBinding:
      position: 5
      prefix: --csp_name
  chemicalShifts:
    type: File
    inputBinding:
      position: 6
      prefix: --cs_file
  pH:
    type: float
    inputBinding:
      position: 7
      prefix: --pH
  temperature:
    type: float
    inputBinding:
      position: 8
      prefix: --temp

outputs: []

Error:

usage$ cwl-runner csp_shiftx.cwl csp_shiftx.yml 
/usr/bin/cwl-runner 1.0.20180302231433
Resolved 'csp_shiftx.cwl' to 'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl'
Tool definition failed initialization:
Tool definition CommentedMap([('class', 'Workflow'), ('requirements', [CommentedMap([('class', 'ScatterFeatureRequirement')])]), ('inputs', [CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/CS_file')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/CS_path')]), CommentedMap([('type', 'File'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/add_hydrogen_script')]), CommentedMap([('type', 'File'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/cfg_file')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/chain')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/cs_predictor')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/outputFormat')]), CommentedMap([('type', 'float'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pH_value')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdbH_file')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdb_filename')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdb_id')]), CommentedMap([('type', 'File'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/py_buildPDB')]), CommentedMap([('type', 'File'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/shift_predict_script')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/task_buildPDB')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/task_insertCS')]), CommentedMap([('type', 'float'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/temp')]), CommentedMap([('type', 'string'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/temp_file')])]), ('outputs', [CommentedMap([('type', CommentedMap([('type', 'array'), ('items', 'File')])), ('outputSource', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbFile', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/pdbH_File', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/CS_file']), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/output_files')])]), ('steps', [CommentedMap([('run', u'file:///local/PycharmProjects/CSPworkflow/code/usage/buildPDB.cwl'), ('scatter', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbID', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbFilename']), ('scatterMethod', 'dotproduct'), ('in', [CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/chain'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/Chain')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/cfg_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/cfgFile')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdb_filename'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbFilename')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdb_id'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbID')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/py_buildPDB'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pyBuildPDB')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/task_buildPDB'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/taskBuildPDB')])]), ('out', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbFile']), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile')]), CommentedMap([('run', u'file:///local/PycharmProjects/CSPworkflow/code/usage/populate_database.cwl'), ('scatter', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/pdbID']), ('in', [CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/chain'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/Chain')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/cfg_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/cfgFile')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/CS_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/chemicalShifts')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/cs_predictor'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/csPredictor')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pH_value'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/pH')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdb_id'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/pdbID')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/py_buildPDB'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/pyInsertCS')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/task_insertCS'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/taskInsertCS')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/temp'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase/temperature')])]), ('out', []), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/populateDatabase')]), CommentedMap([('run', u'file:///local/PycharmProjects/CSPworkflow/code/usage/shiftPredict.cwl'), ('scatter', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/chemical_shiftDir', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/chemical_shiftName']), ('scatterMethod', 'dotproduct'), ('in', [CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/CS_path'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/chemical_shiftDir')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/CS_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/chemical_shiftName')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/outputFormat'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/format')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pH_value'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/pH')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/pdbH_File'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/protonated_pdb')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/shift_predict_script'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/shiftPredict_script')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/temp'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/temperature')])]), ('out', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift/CS_file']), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/predictChemShift')]), CommentedMap([('run', u'file:///local/PycharmProjects/CSPworkflow/code/usage/addHydrogen.cwl'), ('scatter', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/tempFile', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/protonatedPDB']), ('scatterMethod', 'dotproduct'), ('in', [CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/add_hydrogen_script'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/addHydrogen_script')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/create_pdbFile/pdbFile'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/pdb_to_addHydrogens')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/pdbH_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/protonatedPDB')]), CommentedMap([('source', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/temp_file'), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/tempFile')])]), ('out', [u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate/pdbH_File']), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9/protonate')])]), ('id', u'file:///local/PycharmProjects/CSPworkflow/code/usage/csp_shiftx.cwl#subworkflow1/e4a5f7d8-9fd2-40b1-a1ce-3978c1cb5af9')]) failed validation:
csp_shiftx.cwl:57:9:   Source 'CS_path' of type "string" is incompatible
csp_shiftx.cwl:100:13:     with sink 'chemical_shiftDir' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:58:9:   Source 'CS_file' of type "string" is incompatible
csp_shiftx.cwl:101:13:     with sink 'chemical_shiftName' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:92:17:   Source 'pdbH_File' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:99:13:     with sink 'protonated_pdb' of type "File"
csp_shiftx.cwl:50:9:   Source 'pdb_filename' of type "string" is incompatible
csp_shiftx.cwl:77:13:     with sink 'pdbFilename' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:49:9:   Source 'pdb_id' of type "string" is incompatible
csp_shiftx.cwl:76:13:     with sink 'pdbID' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:105:17:   Source 'CS_file' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:114:13:     with sink 'chemicalShifts' of type "File"
csp_shiftx.cwl:49:9:   Source 'pdb_id' of type "string" is incompatible
csp_shiftx.cwl:116:13:     with sink 'pdbID' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:82:17:   Source 'pdbFile' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:89:13:     with sink 'pdb_to_addHydrogens' of type "File"
csp_shiftx.cwl:55:9:   Source 'pdbH_file' of type "string" is incompatible
csp_shiftx.cwl:90:13:     with sink 'protonatedPDB' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:54:9:   Source 'temp_file' of type "string" is incompatible
csp_shiftx.cwl:91:13:     with sink 'tempFile' of type {"items": "string", "type": "array"}
csp_shiftx.cwl:82:17:   Source 'pdbFile' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:68:11:     with sink 'output_files' of type {"type": "array", "items": "File"}
csp_shiftx.cwl:68:11:     source has linkMerge method merge_nested
csp_shiftx.cwl:92:17:   Source 'pdbH_File' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:68:11:     with sink 'output_files' of type {"type": "array", "items": "File"}
csp_shiftx.cwl:68:11:     source has linkMerge method merge_nested
csp_shiftx.cwl:105:17:   Source 'CS_file' of type {"items": "File", "type": "array"} is incompatible
csp_shiftx.cwl:68:11:     with sink 'output_files' of type {"type": "array", "items": "File"}
csp_shiftx.cwl:68:11:     source has linkMerge method merge_nested
3

Hey Levi,

I haven’t looked closely over all the code you’ve posted, but I believe I’m seeing two patterns here:

  1. Nested scattering where you likely don’t want to. For example, you scatter over the string array CS_path in subworkflow1, and you scatter these products (strings) once more in predictChemShift for chemical_shiftDir. shiftPredict.cwl is expecting a string for this parameter, which you have already obtained before your second scatter of chemical_shiftDir. This becomes an error when you introduce your second scatter, because you may only scatter over arrays. Would it make more sense to simply reuse CS_path rather than chemical_shiftDir in predictChemShift?

  2. Not specifying a linkMerge method when you likely want mergeFlattened. This is what leads to the errors mentioning “merge_nested”. pdbFile, pdbH_File, and CS_file are all of type File[] since they are the products of a scattered workflow. This means that when you wrap them in an array in your outputSource, you have an array of arrays, not a single large File array. If you want to merge these arrays into one, then you should specify merge_flattened like:

outputSource: [source1, source2, ...]
linkMerge: merge_flattened

Hopefully this helps.
-Alex

P.S. - you’ve shared quite a bit of code here… I understand that it is difficult to reduce these kinds of problems to a “model example”, but it took a fair amount of time to scroll back and forth through multiple lengthy documents in the small scrolling boxes provided by the forum, and reposting a slightly modified version of the entire codebase made it difficult to ctrl+f my way around. Not trying to be rude but you may have better luck with answers if you take some time to reduce the surface area of your question.