Complex workflow cannot be packed

CWL Questions
1

Hi team,

I have a workflow that generates invalid imports when packed.

From

Release dragen-wgts-dna-pipeline/4.4.4__20251009055750 · umccr/cwl-ica

I have a attached a copy of the zipped workflow and the subsequent packed workflow (compressed)

dragen-wgts-dna-pipeline__4.4.4__20251009055750.zip

dragen-wgts-dna-pipeline__4.4.4__20251009055750.packed.cwl.json.gz

When packed, there are a lot of “$import”: “#dragen-qc-coverage__1.0.0.yaml/dragen-qc-coverage” but the actual schema is not defined anywhere.

Running

cwltool --validate \
  <( \
    wget -q -O- \
      "https://github.com/umccr/cwl-ica/releases/download/dragen-wgts-dna-pipeline%2F4.4.4__20251009055750/dragen-wgts-dna-pipeline__4.4.4__20251009055750.packed.cwl.json.gz" | \
    zcat \
  )

Gives

ERROR Tool definition failed validation:
(139693097291072) (file:///proc/self/fd/11) Validation error in position 4:
  (139693097291072) (file:///proc/self/fd/11) Validation error in field requirements:
    (139693097291072) (file:///proc/self/fd/11) Validation error in position 1:
      (139693097291072) (file:///proc/self/fd/11) Validation error in field types:
        (139693097291072) (file:///proc/self/fd/11) Validation error in position 2:
          Reference '#dragen-qc-coverage__1.0.0.yaml/dragen-qc-coverage' not found in file 'file:///proc/self/fd/11'.

Implications of this are that I cannot make a nice graph describing the workflow or any of the subworkflows.

2

On the contrary, I was able to make an update of the workflow that does pack correctly

Release: Release dragen-wgts-dna-pipeline/4.4.4__20251015010222 · umccr/cwl-ica · GitHub

Zipped: dragen-wgts-dna-pipeline__4.4.4__20251015010222.zip

Packed: dragen-wgts-dna-pipeline__4.4.4__20251015010222.packed.cwl.json.gz

But when I run the non-packed version of this workflow through the ICAv2 Orchestration Engine I get the following errors:

schema_salad.avro.schema.SchemaParseException: The name "dragen-qc-coverage" is already in use.

Full error report can be found at: Schema already in use error · Issue #2167 · common-workflow-language/cwltool

3

Which tool did the packing and what version of the tool is that?

How did you make your fix?

4

Which tool did the packing and what version of the tool is that?

Using cwltool --pack to pack the workflow with version 3.1.20250925164626

sbpack doesn’t work since it cannot handle the nested schema imports.

How did you make your fix?

I was able to make the fix by ensuring there was at least one tool had the offending nested schema as a direct input (rather than a nested input). This involved making dummy tools and then using a when: false on the conditionals of that step so that the tool itself was never run.

It fixed the packing part, but it caused the schema parse exception shown in the GitHub issue 2167.