Cwl with python and python pip requirements

Hey,
I wrote a nice self-containing python in the cwl. How can I include python’s pip requirements: “pip install” before running the script?
I want to keep everything in one script, see below.
Thank you for your help.
Steffen

cwlVersion: v1.0
class: CommandLineTool
baseCommand: ["python", "convert.py"]
inputs:
  in_file:
    type: File
  out_file:
    type: string
requirements:
  InitialWorkDirRequirement:
    listing:
      - entryname: convert.py
        entry: |-
          import numpy as np
          data = np.loadtxt('$(inputs.in_file.path)')
          with open('$(inputs.out_file)','w') as fOut:
            fOut.write("Sum is: "+str(np.sum(data)))
outputs:
  outfile:
    type: File
    outputBinding:
      glob: $(inputs.out_file)

Welcome @SteffenBrinckmann

If your CWL runner support it, then you could add a SoftwareRequirement of numpy

For example, with the CWL reference runner: GitHub - common-workflow-language/cwltool: Common Workflow Language reference implementation you can add the --beta-conda-dependencies flag. If the libraries you need are in PyPI only (or if you prefer PyPI over the Conda registry) then that feature could be extended for a --beta-pip-dependencies flag in the future.

The other option is to add a DockerRequirement either pointing to an image with the dependencies installed, or a dockerFile with an inline recipe using the packages you need.

Sorry I did not post my solution:

- baseCommand: ["sh","script.sh"]
- requirements:
  InitialWorkDirRequirement:
    listing:
      - entryname: script.sh
        entry: |-
          pip3 install -r requirements.txt
          python convert.py

and similarly I add requirements.txt and the python-script as initialWorkDirRequirement.

@mrc Thank you for your suggestion:

  • I am on the PyPi train
  • I want to have the cwl applicable for all runners
    I don’t know of an image that would fit and I am not too much into docker-recipes that I know how to solve it that way. I am a scipt-kid who hopes the sh-commands does work.

Glad you found a solution! I’ve opened up add --beta-pip-dependencies · Issue #1492 · common-workflow-language/cwltool · GitHub for the pip version as requested