Hi,
A command tool requires multiple scripts in the same directory (such as a.R, b.py, c.dat for run.sh). Could anyone please show me how to stage those dependency files for baseCommand? Thanks!
Hi,
A command tool requires multiple scripts in the same directory (such as a.R, b.py, c.dat for run.sh). Could anyone please show me how to stage those dependency files for baseCommand? Thanks!
Hi @hubentu
There are a number of ways to do this. Here’s what I would do in roughly descending order of preference:
inputs:
script:
type: File
default:
class: File
location: run.sh
secondaryFiles:
- class: File
location: a.R
- class: File
location: b.py
- class: File
location: c.dat
arguments: [$(inputs.script)]
inputs:
script:
type: Directory
default:
class: Directory
location: myscripts
arguments: [$(inputs.script)/run.sh]
This puts all the files in the working directory (which is also the output directory).
inputs:
script:
type: File
default:
class: File
location: run.sh
a:
type: File
default:
class: File
location: a.R
b:
type: File
default:
class: File
location: b.py
c:
type: File
default:
class: File
location: c.dat
requirements:
InitialWorkingDirRequirement:
listing:
- $(inputs.script)
- $(inputs.a)
- $(inputs.b)
- $(inputs.c)
arguments: [$(inputs.script)]
Note: for (1) and (2) the working directory will not be the same as the directory where the script is located, so you may need a little bit of code to figure out where the script was placed to in order to find the other files it depends on. In shell you can do dirname $0
.
Hope this helps!
Thanks a lot for the explicit solutions. This is very helpful.
Hi @tetron,
Thanks again for the answers. I have tried the 3 solutions. Only InitialWorkDirRequirement
works for me.
My test script run.sh
:
#!/bin/sh
cat a.dat
cwlVersion: v1.0
class: CommandLineTool
requirements:
InitialWorkDirRequirement:
listing:
- $(inputs.src)
arguments:
- $(inputs.src)
inputs:
src:
type: File
default:
class: File
location: run.sh
secondaryFiles:
- class: File
location: a.dat
outputs: []
stdout: out.txt
This one requires the InitialWorkDirRequirement
to work. Otherwise, a.dat
couldn’t be found.
cwlVersion: v1.0
class: CommandLineTool
arguments:
- $(inputs.src.dirname)/$(inputs.src.basename)/run.sh
inputs:
src:
type: Directory
default:
class: Directory
location: ./
outputs: []
The upper directory of the inputs will be mapped to ‘$(inputs.src)’, so the a.dat
could not be found in the work directory.
INFO [job run_dir.cwl] /tmp/vp3nic_i$ /tmp/tmpxfakvow_/stg49989a4b-73bd-4117-8d35-c979e24c97cc/test/run.sh
cat: a.dat: No such file or directory
cwlVersion: v1.0
class: CommandLineTool
requirements:
InitialWorkDirRequirement:
listing:
- $(inputs.src)
- $(inputs.a)
arguments:
- $(inputs.src)
inputs:
src:
type: File
default:
class: File
location: run.sh
a:
type: File
default:
class: File
location: a.dat
outputs: []
stdout: out.txt
This one works perfectly.
Thanks!
Your sample script requires files to be in the current working directory, not that same directory as the script. This is why some of @tetron’s solutions didn’t work for you.
Here is an example script that requires the current working directory:
#!/bin/sh
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
# from https://stackoverflow.com/a/246128
cat ${DIR}/a.dat
Is this case, all three solutions provided should work.
I’m glad you found a method that works for you!