# CWL Questions **URL:** https://cwl.discourse.group/c/q-and-a/5.md?page=1 [Latest](https://cwl.discourse.group/latest.md) · [Categories](https://cwl.discourse.group/categories.md) **Page:** 2 --- ## ["Singularity exec" vs "docker run"](https://cwl.discourse.group/t/singularity-exec-vs-docker-run/956)
**Author:** [@Samee](https://cwl.discourse.group/u/Samee)\ **Replies:** 6\ **Last updated:** [November 12, 2024, 3:38pm UTC](https://cwl.discourse.group/t/singularity-exec-vs-docker-run/956 "2024-11-12T15:38:48Z")
Hi there! New CWL/Toil user here. We have a workflow where we noticed that depending on whether singularity or docker container choice it uses exec vs run. I am trying to understand what was the rational behind this dif… --- ## [How to gather output files from subdirectories](https://cwl.discourse.group/t/how-to-gather-output-files-from-subdirectories/958)
**Author:** [@ndonyapour](https://cwl.discourse.group/u/ndonyapour)\ **Replies:** 1\ **Last updated:** [November 11, 2024, 9:11am UTC](https://cwl.discourse.group/t/how-to-gather-output-files-from-subdirectories/958 "2024-11-11T09:11:15Z")
Hello, I have a tool that downloads a database with image files saved in subdirectories within the output folder. I’m writing a CWL for this tool to gather the files from these subdirectories, but it fails because input… --- ## [How to run a bash command within a tool?](https://cwl.discourse.group/t/how-to-run-a-bash-command-within-a-tool/952)
**Author:** [@kvertacnik](https://cwl.discourse.group/u/kvertacnik)\ **Replies:** 3\ **Last updated:** [November 5, 2024, 11:12pm UTC](https://cwl.discourse.group/t/how-to-run-a-bash-command-within-a-tool/952 "2024-11-05T23:12:08Z")
I have a tool that utilizes a singularity container, however, I am on an HPC where the singularity module must be loaded prior to using a container. The command for this is “module load singularity”. I’m trying to load … --- ## [How to use arrays as input](https://cwl.discourse.group/t/how-to-use-arrays-as-input/954)
**Author:** [@kvertacnik](https://cwl.discourse.group/u/kvertacnik)\ **Replies:** 3\ **Last updated:** [November 5, 2024, 2:41pm UTC](https://cwl.discourse.group/t/how-to-use-arrays-as-input/954 "2024-11-05T14:41:59Z")
Apologies for the basic question. I have gff and genome file pairs that I would like to process together, however, the number of file pairs will vary. So I’m trying to input arrays of file pairs. The error I’m getting is… --- ## [Using a local Dockerfile based on renv.lock](https://cwl.discourse.group/t/using-a-local-dockerfile-based-on-renv-lock/941)
**Author:** [@Brilator](https://cwl.discourse.group/u/Brilator)\ **Replies:** 1\ **Last updated:** [November 5, 2024, 1:07pm UTC](https://cwl.discourse.group/t/using-a-local-dockerfile-based-on-renv-lock/941 "2024-11-05T13:07:33Z")
Hi everyone, in order to use and share an R script with CWL, I am trying to create and reference a Dockerfile based on the R package renv; mostly following the renv + docker instructions: (Using renv with Docker • renv … --- ## [Cwltool/provenance worfklow.cwl#main vs workflow.cwl](https://cwl.discourse.group/t/cwltool-provenance-worfklow-cwl-main-vs-workflow-cwl/948)
**Author:** [@FabriceBrito](https://cwl.discourse.group/u/FabriceBrito)\ **Replies:** 0\ **Last updated:** [September 2, 2024, 3:00pm UTC](https://cwl.discourse.group/t/cwltool-provenance-worfklow-cwl-main-vs-workflow-cwl/948 "2024-09-02T15:00:25Z")
I think I ran into a small issue in cwltool. cwltool --provenance /tmp/provenance workflow.cwl#main params.yml works fine, instead cwltool --provenance /tmp/provenance app-package.cwl params.yml throws an exception: E… --- ## [ExpressionTools loadContents how to load from array](https://cwl.discourse.group/t/expressiontools-loadcontents-how-to-load-from-array/947)
**Author:** [@CloXD](https://cwl.discourse.group/u/CloXD)\ **Replies:** 0\ **Last updated:** [August 29, 2024, 12:30pm UTC](https://cwl.discourse.group/t/expressiontools-loadcontents-how-to-load-from-array/947 "2024-08-29T12:30:56Z")
Hello! I’m trying to concatenate the content of some small files using the ExpressionTool, but it seems that the loadContents doesn’t work on arrays, despite in the doc it includes arrays of files. Here’s a small examp… --- ## [How to use the Docker installed libraries in a cwl workflow?](https://cwl.discourse.group/t/how-to-use-the-docker-installed-libraries-in-a-cwl-workflow/944)
**Author:** [@rupert\_may](https://cwl.discourse.group/u/rupert_may)\ **Replies:** 4\ **Last updated:** [August 19, 2024, 1:40pm UTC](https://cwl.discourse.group/t/how-to-use-the-docker-installed-libraries-in-a-cwl-workflow/944 "2024-08-19T13:40:15Z")
Hello, I am creating a cwl workflow, where I am writing the steps to run. The first step is regarding a Docker container, which is used to install some Python libraries. Then, I want the other cwl files to point to the … --- ## [Workflow fails when run without --cachedir](https://cwl.discourse.group/t/workflow-fails-when-run-without-cachedir/942)
**Author:** [@Brilator](https://cwl.discourse.group/u/Brilator)\ **Replies:** 2\ **Last updated:** [August 13, 2024, 11:39am UTC](https://cwl.discourse.group/t/workflow-fails-when-run-without-cachedir/942 "2024-08-13T11:39:32Z")
Hi, can someone help me understand allocating resources to workflows? I’m trying to run a rather simple workflow, scattered over an array of inputs (i.e. samples). Per sample the workflow creates ca. 10GB temporary dat… --- ## [Using a workflow input in a step\`s valueFrom](https://cwl.discourse.group/t/using-a-workflow-input-in-a-step-s-valuefrom/940)
**Author:** [@Brilator](https://cwl.discourse.group/u/Brilator)\ **Replies:** 2\ **Last updated:** [August 5, 2024, 6:35am UTC](https://cwl.discourse.group/t/using-a-workflow-input-in-a-step-s-valuefrom/940 "2024-08-05T06:35:49Z")
Hi CWL folks, I’m trying to use an input from a workflow’s inputs section as a variable in a valueFrom expression. The following works, but hardcodes to extract a substring of 11 characters from nameroot: cwlVersion: … --- ## [Graphical User Interface like Rabix Composer](https://cwl.discourse.group/t/graphical-user-interface-like-rabix-composer/934)
**Author:** [@Echelon\_Bioanalytics](https://cwl.discourse.group/u/Echelon_Bioanalytics)\ **Replies:** 0\ **Last updated:** [July 4, 2024, 1:36am UTC](https://cwl.discourse.group/t/graphical-user-interface-like-rabix-composer/934 "2024-07-04T01:36:36Z")
Hello, I have been working with Rabix Composer for a while now. It works fine but does not have the functionality to include version 1.1. Also since it is out of development, several bugs are left unattended. I was wond… --- ## [Collect input files by name from directories](https://cwl.discourse.group/t/collect-input-files-by-name-from-directories/932)
**Author:** [@Brilator](https://cwl.discourse.group/u/Brilator)\ **Replies:** 0\ **Last updated:** [July 3, 2024, 2:58pm UTC](https://cwl.discourse.group/t/collect-input-files-by-name-from-directories/932 "2024-07-03T14:58:28Z")
Hi everyone, I’m trying to collect input files for a workflow step from various directories into one directory (as required input for the next step). The files should match a name pattern (i.e. be of a specific file ty… --- ## [.cwl workflow find file in subdirectories of an input directory](https://cwl.discourse.group/t/cwl-workflow-find-file-in-subdirectories-of-an-input-directory/191)
**Author:** [@cccnrc](https://cwl.discourse.group/u/cccnrc)\ **Replies:** 7\ **Last updated:** [July 3, 2024, 7:12am UTC](https://cwl.discourse.group/t/cwl-workflow-find-file-in-subdirectories-of-an-input-directory/191 "2024-07-03T07:12:17Z")
I have output files from a previous step organized as files inside subdirectories of an output directory. Such as: /home/enrico/Dropbox/NY/app/GATK\_CNV\_germline/scatterIntervals/scattered\_hs37d5.preprocessed.300bp.prima… --- ## [How to get input name from a tool parsed using \`cwl\_utils.parser.load\_document\_by\_uri\`](https://cwl.discourse.group/t/how-to-get-input-name-from-a-tool-parsed-using-cwl-utils-parser-load-document-by-uri/922)
**Author:** [@sphuber](https://cwl.discourse.group/u/sphuber)\ **Replies:** 2\ **Last updated:** [July 1, 2024, 10:20am UTC](https://cwl.discourse.group/t/how-to-get-input-name-from-a-tool-parsed-using-cwl-utils-parser-load-document-by-uri/922 "2024-07-01T10:20:41Z")
Hello everyone, I am trying out the cwl\_utils package to parse CWL documents. Consider the following tool.cwl: cwlVersion: v1.0 class: CommandLineTool baseCommand: echo inputs: message: type: string inputBind… --- ## [Cwltool uses docker --mounts](https://cwl.discourse.group/t/cwltool-uses-docker-mounts/929)
**Author:** [@tjvneste](https://cwl.discourse.group/u/tjvneste)\ **Replies:** 1\ **Last updated:** [July 1, 2024, 10:17am UTC](https://cwl.discourse.group/t/cwltool-uses-docker-mounts/929 "2024-07-01T10:17:13Z")
I’m trying to implement tests for my cwl code in bitbucket pipelines. Therefore I need to run the cwl code using: cwltool main.cwl inputs.yml Although this works locally, if you run this in the bitbucket pipeline you g… --- ## [CWL - specifying Directory outputs as an input](https://cwl.discourse.group/t/cwl-specifying-directory-outputs-as-an-input/107)
**Author:** [@skanwal](https://cwl.discourse.group/u/skanwal)\ **Replies:** 13\ **Last updated:** [May 15, 2024, 6:55pm UTC](https://cwl.discourse.group/t/cwl-specifying-directory-outputs-as-an-input/107 "2024-05-15T18:55:55Z")
Hello, I am working on a tool definition that requires specifying an input parameter for the outputs location. I have managed to write the following tool definition, that works as expected when testing with cwltool. c… --- ## [How to have configure secondaryFiles to recognize two different patterns for the same file](https://cwl.discourse.group/t/how-to-have-configure-secondaryfiles-to-recognize-two-different-patterns-for-the-same-file/729)
**Author:** [@steve](https://cwl.discourse.group/u/steve)\ **Replies:** 2\ **Last updated:** [May 18, 2024, 8:27pm UTC](https://cwl.discourse.group/t/how-to-have-configure-secondaryfiles-to-recognize-two-different-patterns-for-the-same-file/729 "2024-05-18T20:27:29Z")
We have a lot of legacy systems that output files in a naming format like this; Sample1.bam Sample1.bai And we have some other systems that have the same files named like this; Sample1.bam Sample1.bam.bai I need to a… --- ## [Example CWL FastAPI execution in Python](https://cwl.discourse.group/t/example-cwl-fastapi-execution-in-python/919)
**Author:** [@jzvolensky](https://cwl.discourse.group/u/jzvolensky)\ **Replies:** 2\ **Last updated:** [May 18, 2024, 6:05pm UTC](https://cwl.discourse.group/t/example-cwl-fastapi-execution-in-python/919 "2024-05-18T18:05:07Z")
Hello everyone, I am currently working on a little testing project of creating a smalled dockerized FastAPI service for executing CWL workflows via OGC API. I am storing CWL workflows in an sqlite db and would like to e… --- ## [CPU allocation from Slurm](https://cwl.discourse.group/t/cpu-allocation-from-slurm/910)
**Author:** [@tikk3r](https://cwl.discourse.group/u/tikk3r)\ **Replies:** 4\ **Last updated:** [May 13, 2024, 5:21pm UTC](https://cwl.discourse.group/t/cpu-allocation-from-slurm/910 "2024-05-13T17:21:31Z")
I was wondering how cwltool determines how many cores it has available to distribute to jobs. The only bit to go on that I could find was in cwltool/cwltool/executors.py, where I see a reference to psutil.cpu\_count(), bu… --- ## [Creating config JSON file from a record](https://cwl.discourse.group/t/creating-config-json-file-from-a-record/172)
**Author:** [@stain](https://cwl.discourse.group/u/stain)\ **Replies:** 3\ **Last updated:** [March 19, 2024, 10:38pm UTC](https://cwl.discourse.group/t/creating-config-json-file-from-a-record/172 "2024-03-19T22:38:10Z")
Hi, I am trying to formalize a JSON input taken as input by biobb command line tools. At the moment we have received these as strings, which is OK, however it means the values are always hardcoded and can’t come from the… --- ## [How to fail-fast during parallel scatter](https://cwl.discourse.group/t/how-to-fail-fast-during-parallel-scatter/868)
**Author:** [@tate](https://cwl.discourse.group/u/tate)\ **Replies:** 4\ **Last updated:** [February 4, 2024, 12:49am UTC](https://cwl.discourse.group/t/how-to-fail-fast-during-parallel-scatter/868 "2024-02-04T00:49:06Z")
Hello, I have a workflow that I execute using cwltool --parallel. The workflow has a scatter step that is both time and memory intensive. Sometimes during early runs of the workflow, before users have their environment … --- ## [Parallel execution of tasks. Is there a limit?](https://cwl.discourse.group/t/parallel-execution-of-tasks-is-there-a-limit/867)
**Author:** [@Konstantinos\_Mavromm](https://cwl.discourse.group/u/Konstantinos_Mavromm)\ **Replies:** 3\ **Last updated:** [February 2, 2024, 4:14pm UTC](https://cwl.discourse.group/t/parallel-execution-of-tasks-is-there-a-limit/867 "2024-02-02T16:14:02Z")
HI, I have a workflow that has several tasks that can run concurrently. S1 → S2 → C1/C2/C3 → S3 → C3/C5/C6 → S4 Cx = concurrent task Px= parallel task I have noticed that when I execute this workflow only two of the… --- ## [How adapted to batch analysis is CWL?](https://cwl.discourse.group/t/how-adapted-to-batch-analysis-is-cwl/864)
**Author:** [@blaurent](https://cwl.discourse.group/u/blaurent)\ **Replies:** 2\ **Last updated:** [January 16, 2024, 10:10am UTC](https://cwl.discourse.group/t/how-adapted-to-batch-analysis-is-cwl/864 "2024-01-16T10:10:55Z")
Hi, Kind of discovering CWL. So far I like a lot of things about it, but I have to say that I am not able to evaluate how adapted or unadapted it is to my requirements. I want to run a batch (potentially tens) of anal… --- ## [Cwltool.main text arguments a hard requirement for provenance?](https://cwl.discourse.group/t/cwltool-main-text-arguments-a-hard-requirement-for-provenance/862)
**Author:** [@davidjsherman](https://cwl.discourse.group/u/davidjsherman)\ **Replies:** 2\ **Last updated:** [January 13, 2024, 7:52am UTC](https://cwl.discourse.group/t/cwltool-main-text-arguments-a-hard-requirement-for-provenance/862 "2024-01-13T07:52:54Z")
It seems that the text of the command-line arguments is always required when requesting provenance, even when providing preparsed arguments, and even though setup\_provenance will happily skip them if absent. Why is this … --- ## [How to use CUDARequirement?](https://cwl.discourse.group/t/how-to-use-cudarequirement/861)
**Author:** [@marveso](https://cwl.discourse.group/u/marveso)\ **Replies:** 2\ **Last updated:** [January 8, 2024, 9:20am UTC](https://cwl.discourse.group/t/how-to-use-cudarequirement/861 "2024-01-08T09:20:35Z")
Hello guys, I’d appreciate if you could help me out with assigning a GPU to a docker container within CWL. I’ve been searching how to do this for hours, but I couldn’t manage to make it work… I’m trying to use the cwlto… --- ## [Running docker from a CommandLineTool](https://cwl.discourse.group/t/running-docker-from-a-commandlinetool/836)
**Author:** [@locos](https://cwl.discourse.group/u/locos)\ **Replies:** 3\ **Last updated:** [January 5, 2024, 1:23pm UTC](https://cwl.discourse.group/t/running-docker-from-a-commandlinetool/836 "2024-01-05T13:23:05Z")
I have the same problem, the code below doesn’t work: baseCommand: \[docker, run, -v, /data:/data, --user, $(id -u):$(id -g), image1:v1\] --- ## [Build full command with CWL parsing](https://cwl.discourse.group/t/build-full-command-with-cwl-parsing/855)
**Author:** [@peterg1t](https://cwl.discourse.group/u/peterg1t)\ **Replies:** 4\ **Last updated:** [January 4, 2024, 7:42pm UTC](https://cwl.discourse.group/t/build-full-command-with-cwl-parsing/855 "2024-01-04T19:42:42Z")
Hi all, This is more of a developer question than a user question. A full command is composed of a base command (baseCommand), a series of inputs with positions defined in inputBindings and arguments also with position… --- ## [dockerFile contents vs dockerFile...file](https://cwl.discourse.group/t/dockerfile-contents-vs-dockerfile-file/221)
**Author:** [@qwofford](https://cwl.discourse.group/u/qwofford)\ **Replies:** 6\ **Last updated:** [December 16, 2023, 11:53am UTC](https://cwl.discourse.group/t/dockerfile-contents-vs-dockerfile-file/221 "2023-12-16T11:53:22Z")
Hello, I’d like to understand why the dockerFile hint/requirement refers to the contents of a file vs the file itself. Thanks, Quincy --- ## [Specifying custom hints for specific workflow engines](https://cwl.discourse.group/t/specifying-custom-hints-for-specific-workflow-engines/835)
**Author:** [@alexiswl](https://cwl.discourse.group/u/alexiswl)\ **Replies:** 5\ **Last updated:** [December 13, 2023, 5:50pm UTC](https://cwl.discourse.group/t/specifying-custom-hints-for-specific-workflow-engines/835 "2023-12-13T17:50:09Z")
On a similar theme to Using schema.org attributes to define compatible workflow engines, ICAv2 has three options for its scratch storage space (small, medium and large), (1.2 Tb, 2.4 Tb or 7.2 Tb respectively). The scra… --- ## [Using schema.org attributes to define compatible workflow engines](https://cwl.discourse.group/t/using-schema-org-attributes-to-define-compatible-workflow-engines/834)
**Author:** [@alexiswl](https://cwl.discourse.group/u/alexiswl)\ **Replies:** 5\ **Last updated:** [December 8, 2023, 10:38pm UTC](https://cwl.discourse.group/t/using-schema-org-attributes-to-define-compatible-workflow-engines/834 "2023-12-08T22:38:18Z")
Hello, I am writing a suite of workflows in our cwl-ica repository. These workflows were written for ICAv1 (which is being deprecated). We are moving to using ICAv2 along with testing out other CWL compatible workflow … [Previous page](https://cwl.discourse.group/c/q-and-a/5.md) [Next page](https://cwl.discourse.group/c/q-and-a/5.md?page=2)