Tool 'activate'

Hello, CWL community!

Question, in short: Is there a clean way to use cwltool or some other runner to go through execution environment setup steps, without executing the actual base command?

Some context. What I would like to achieve is starting an interactive shell in the execution environment that the tool’s command would be run in, analogous to a venv’s “activate”. Conceptually, I’m thinking of a feature where the runner skips the input/output validation and binding, still prepares the workdir and salient requirements, and runs bash or similar in interactive mode.

To offer some more background, our team has been relying on containers to provide ‘easy’ access pretty gnarly software tools. As we’re moving to rely more on CWL, we have also started to move some of the environment setup out of the dockerfile and into the CWL tool description, relying on the cwltool runner to do some of the prep (e.g. InitialWorkDirRequirement to instruct the runner to do the mount/bind). That also means that the previous functionality of ‘activating’ a tool by simply attaching the user to the underlying tool container image no longer works, as various setup logic has been moved into the cwl runner.

cwltool has overrides for requirements and targets for workflow steps, but I don’t see a way to make that work for what I’m trying to achieve. Am I missing something obvious? Is there some easy way to make this work beyond: a) maintaining a parallel cwl tool description (not ideal) or b) adding the feature to cwltool ourselves?

Thanks!

2 Likes

Welcome @yvdriess !

This is a great idea, and not one that I’ve heard of before!

Unless someone else knows of an external tool to do this, one would need to

  1. enhance cwltool.job._job_popen() to not redirect stdin / stdout / stderr
  2. insert bash or other shell instead of the baseCommand and arguments and other inputBindings in cwltool.process.Process._init_job().

This new feature would combine well with --single-step or --single-process for selecting which step to enter the interactive environment for.

1 Like