Passing slurm job parameters

CWL Questions
1

I feel like I’m missing something obvious.

I am using cromwell to run cwl on an HPC cluster. The cluster uses slurm for job management. I want to be able to adjust the resources requested of slurm, dependent on what different tasks within the cwl workflow. i.e. I would like to request more memory, time and cpus when running bwa than when I am running a samtools sort. To me, this seems like a fairly basic requirement, there must be documentation around this somewhere surely?

Cromwell allows you to specify a config file that overrules the defaults, creating the slurm job file. I can’t find anything that tells me how to get information from the cwl file, to that config file though, which means that at the moment, I can only specify a single set of slurm job requirements with no ability to customize.

I thought it might have involved runtime variables - but a) I don’t appear to be able to pass them to the cromwell config file and b) I can’t see a way to pass variables that I define, like how much time I want the job to reserve.

Cheers
Ben.

2

Hi @tirohia you should be using https://www.commonwl.org/v1.1/CommandLineTool.html#ResourceRequirement then it is the job of the CWL runner (Cromwell in this case) to translate that into the slurm resource request.

I checked and I see that ResourceRequirement is missing from the User Guide, so that’s probably why you are confused. We have a grant starting soon to help fund improvements to CWL documentation so hopefully it will get some attention in the next few months.

3

Can you use custom ResourceRequirements? One of the parameters out implementation of slurm requires is a maximum run time. Without it, it defaults to an hour.

I’m having the additional problem of cromwell not picking up the ResourceRequirements, but from what you say, that’s a cromwell issue?

4

I would be surprised if Cromwell did not support ResourceRequirement but I haven’t ever used it so I cannot say for sure.

CWL v1.1 adds a ToolTimeLimit to set the runtime. However the implementation you are using needs to support it. I have not checked recently but I do not believe Cromwell supports CWL v1.1. You might want to look at Toil or Arvados which both support Slurm.

1 Like
5

Worse case you can edit the Cromwell slurm template to specify a large maximum wall time like 24 hours. But it would better if Cromwell supported the ToolTimeLimit

1 Like