matmanc
February 22, 2021, 2:10pm
1
Hi, I am running in the following issue. I have a list of directory that I would like to modify as input.
To do so I add them in the InputWorkDirRequirement but for some reason it only works for a single directory.
This is my cwl workflow
class: CommandLineTool
cwlVersion: v1.1
id: test_multiple_dir_input
baseCommand:
- /bin/bash
- script.sh
inputs:
- id: msin
type: 'Directory[]'
outputs:
- id: std_out
type: File
outputBinding:
glob: test_multiple_dir_input.log
label: test_multiple_dir_input
requirements:
- class: DockerRequirement
dockerPull: 'lofareosc/prefactor:HBAcalibrator'
- class: InitialWorkDirRequirement
listing:
- entry: $(inputs.msin)
writable: true
- entryname: script.sh
entry: |
#!/bin/bash
ls -lR
rm test1/dude.txt
- class: InlineJavascriptRequirement
stdout: test_multiple_dir_input.log
and this is my input file
{
"msin": [{"class": "Directory", "path":"test1"}, {"class": "Directory", "path":"test2"}]
}
the directory listing is the following:
[15:10:07] mmancini:multiple_folders $ ls -lR test1 test2
test1:
total 0
-rw-rw-r-- 1 mmancini mmancini 0 Feb 22 15:10 dude.txt
test2:
total 0
This is obviously a minimum snippet to recreate the problem.
To execute the workflow I am using cwltool with version 3.0.20210124104916
steve
March 2, 2021, 5:06pm
2
Not sure if its exactly the same problem but this sounds kinda similar to the issue I was having here;
Due to the issue described here where input files with the same basename cause errors with a directory InitialWorkDirRequirement, I need to find a way to generate unique filenames that will not conflict with each other.
Ideally, I would just use a Javascript expression inside my CWL to accomplish this, but I could not get any of these suggested methods to generate random strings to work (or they required huge amounts of custom code too large to easily use across many CWL’s).
I did come up with…
You might be able to modify the solutions there to stage a directory of directories
mrc
March 5, 2021, 2:49pm
3
Hello @matmanc
I am able to reproduce your result; can you open an issue at https://github.com/common-workflow-language/cwltool/issues/new ?
INFO /home/michael/cwltool/env3.9/bin/cwltool 3.0.20210124104916
INFO Resolved 'calibrator.cwl' to 'file:///home/michael/cwltool/calibrator.cwl'
INFO [job test_multiple_dir_input] /tmp/g63y6gd4$ docker \
run \
-i \
--mount=type=bind,source=/tmp/g63y6gd4,target=/EIweox \
--mount=type=bind,source=/tmp/ajilqq15,target=/tmp \
--mount=type=bind,source=/home/michael/cwltool/test1,target=/EIweox/test1,readonly \
--mount=type=bind,source=/home/michael/cwltool/test2,target=/EIweox/test2,readonly \
--workdir=/EIweox \
--read-only=true \
--net=none \
--log-driver=none \
--user=1000:1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/EIweox \
--cidfile=/tmp/px3lec2s/20210305154829-021485.cid \
lofareosc/prefactor:HBAcalibrator \
/bin/bash \
script.sh > /tmp/g63y6gd4/test_multiple_dir_input.log
rm: cannot remove 'test1/dude.txt': Read-only file system
INFO [job test_multiple_dir_input] Max memory used: 0MiB
WARNING [job test_multiple_dir_input] completed permanentFail