dockerFile contents vs dockerFile...file

CWL Questions
1

Hello,

I’d like to understand why the dockerFile hint/requirement refers to the contents of a file vs the file itself.

Thanks,
Quincy

2

This is basically a mistake. When it was added we didn’t think through all the ways one would want to use it. It way it ought to work is to allow a File object referring to the Dockerfile that can have secondaryFiles, then it would stage the Dockerfile to a directory like it does now and build the docker image from there, but you would also be able to copy things into the image.

However, if you don’t need to copy things into the image, you can use the $include construct, e.g.

DockerRequirement:
  dockerFile: {$include: "MyDockerfile"}

The $include loads text inline in a preprocessing step.

3

Thanks for the reminder @qwofford, I’ve suggested that the fix @tetron mentioned be implemented for the next release of CWL: https://github.com/common-workflow-language/common-workflow-language/issues/294 (this is no guarantee, however)

4

Thanks for bumping this issue. I am working on a workflow management system for HPC, and we have many non-trivial Dockerfiles that would benefit from referencing a file object rather than Dockerfile contents. I’m implementing the build stage of the workflow system, and while we are trying to keep it CWL-compliant, this issue complicates my work. Any Dockerfile that is longer than a few lines becomes cumbersome.

Here is an example of a Dockerfile and associated repository that I would like to support: https://github.com/eugeneswalker/pantheon-containers/blob/46e424d9ca00baa30fe8bad9324c5a589ed22aff/Dockerfile

I would really like to see either the dockerFile spec description change to reference a file object, or see the spec name change from dockerFile to dockerFileContents. I’m happy to discuss further or provide more examples that would benefit from this change to the specification language.

1 Like
5

I see that CWL v1.2 documentation still references dockerFile contents vs a dockerFile stored on some file share. Is this still possible to be included in CWL v1.3? Since our Dockerfiles are typically quite long, including the dockerfile contents vs a reference to a Dockerfile path makes the CW very difficult to parse visually.

ref: Common Workflow Language (CWL) Command Line Tool Description, v1.2

6

Hello @qwofford. This is on the wishlist for the next version of CWL, yes.

Currently we are prioritising cleanups for a CWL v1.2.1 release and we have yet to begin work on CWL v1.3. If you are able to contribute, then please let us know and I can help you implement this.

7

I’ve added https://github.com/common-workflow-language/cwl-v1.3/issues/17 to our CWL v1.3 wishlist