Hi,
Is there a way to pass an extra option to the docker run command that cwltool runs? For instance I want to increase the size of the /dev/shm size in the container at runtime like this:
docker run -it --shm-size=256m oracle11g /bin/bash
Thanks!
Hmm, I don’t think that’s possible at the moment. I think you can use parameters to cwltool that would be translated in --memory, --cpus, but not shared memory. Let’s wait to see if others are aware of some way of achieving it.
That’s not a in the CWL reference runner (cwltool), as it would hurt portability.
Can you share more about why this option is needed?
@wshands did you ever find a solution to this? I also need to add the --shm-size switch to my docker run command.
For reference as to when this is needed: I have a Python process running inside the container that needs to allocate shared memory larger than the very small default value that docker allocates. Ideally I’d like to be able to specify this requirement under DockerRequirement as part of the CWL file itself.
@mjm I see that the OCI runtime spec has a MaxShmMemory concept as a Solaris specific extension: https://github.com/opencontainers/runtime-spec/blob/494a5a6aca782455c0fbfc35af8e12f04e98a55e/specs-go/config.go#L456
The maximum amount of shared memory allowed for this container.
Anyhow, --shm-size seems to be implemented by many Docker-like software container engines, even though the concept is not part of the OCI spec; so it could be a good candidate for a CWL extension to represent it (and possibly be incorporated into CWL v1.3)
For anyone who wants this, I would recommend opening an issue on the CWL reference runner repository and we’ll help you write an extension and implement it.
I was able to set the global default default-shm-size using the following guide. This satisfies my needs for the moment, but I could see others (or myself in the future) needing more fine control on a per-workflow basis.
1 Like