# Access envDef of Workflow in CWLtool

**URL:** <https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572>\
**Category:** CWL Questions\
**Created:** [March 2, 2022, 11:15pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572 "2022-03-02T23:15:12Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 2, 2022, 11:15pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/1 "2022-03-02T23:15:12Z")

</div>

Hello,  
I would like to know whether is there any way to access the Environment Variable defined in workflow from inside a tool/step of the workflow without explicitly passing it as an argument?

In the CWL tutorials, the only information to define an Env Variable and access it by usually passing it as an argument/input?

But I would like to access it without explicitly passing the variable to a tool. That should be possible as it is there in the environment, but I don’t know how to access it.  
Any help would be appreciated on this. Thank you in advance!

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 3, 2022, 11:39am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/2 "2022-03-03T11:39:45Z")

</div>

Do you have a `EnvVarRequirement` at the workflow level?

Then that `EnvVarRequirement` will automatically be present on all steps and sub-steps, unless they define their own `EnvVarRequirement`.

Accessing the actual value is not possible except for making a `CommandLineTool` that does `echo $MY_ENV_VAR` and captures the output.

```auto
cwlVersion: v1.0
class: CommandLineTool
requirements:
  InlineJavascriptRequirement: {}
  ShellCommandRequirement: {}
 
inputs: []

baseCommand: echo 

arguments:
 - valueFrom: $MY_ENV_VAR
   shellQuote: false

stdout: result.txt

outputs:
  MY_ENV_VAR_value:
     type: string
     outputBinding:
       glob: result.txt
       loadContents: true
       outputEval: $(self[0].contents)

```

In general I would suggest not defining an `EnvVarRequirement` at the workflow level. Can you explain more about your goal in doing that?

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 3, 2022, 11:42am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/3 "2022-03-03T11:42:46Z")

</div>

I have a variable that I need to pass to more than half of the steps/tools, so I thought it might be a better idea to define it as ENV\_VAR and access it directly from tools.

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 3, 2022, 11:47am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/4 "2022-03-03T11:47:50Z")

</div>

> [@HrishiDhondge](#):
>
> I have a variable that I need to pass to more than half of the steps/tools, so I thought it might be a better idea to define it as ENV\_VAR and access it directly from tools.

Then that should be an explicit input for each step.

In chat, you used the example of an output directory. That concept is not needed for CWL as any CWL runner must manage your output directories for you.

If you need to know the current output directory, use `$(runtime.outdir)`

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 3, 2022, 12:41pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/5 "2022-03-03T12:41:07Z")

</div>

Thank you for the prompt answer.  
I have an output directory along with a few more directories, I am working with the splitting of PDB files, so I have one directory for complete PDB files (pdbDir) and another directory for split PDB files (sepDir).  
The pdbDir, has to be there to split the PDB files and return output in sepDIr.  
Returning output can be done using --outdir option, but passing pdbDir to several steps/tools in workflow is a must.  
So I was wondering if there is a simple solution to inherit and access those ENV\_VAR defined at workflow level?

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 3, 2022, 1:26pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/6 "2022-03-03T13:26:34Z")

</div>

Do any of your steps need multiple PDB files and for them to be in the same directory? Then you should have an output of `type: Directory`; in CWL we do not pass file/directory paths around as `string`s, as that would break during distributed execution when there is not a shared filesystem.

Or are you thinking more about presenting the final outputs? In that case, the CWL way would be to have individual `type: File` outputs from steps. Possibly combining them into a directory later, if truly needed.

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 3, 2022, 2:49pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/7 "2022-03-03T14:49:19Z")

</div>

Hello again,

> [@mrc](#):
>
> If you need to know the current output directory, use `$(runtime.outdir)`

```auto
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool

requirements:
  InlineJavascriptRequirement: {}
  InitialWorkDirRequirement:
    listing:
      - $(inputs.src)
inputs:
  src:
    type: File
    inputBinding:
      position: 1
      valueFrom: $(self.basename)

outputs:
  classfile:
    type: stdout

baseCommand:
  - echo
  - $(runtime.outdir)

```

This prints $(runtime.outdir) as string and not its value; And below code says the baseCommand` field is not valid because tried array of but  
try.cwl:24:5: item is invalid because the value is not string

```auto
baseCommand:
  - echo
  - valueFrom: $(runtime.outdir)

```

- And in addition to this I would like to pass the file as argument/input only if it exists because if file does not exists it gives me error like  
[Errno 2] No such file or directory

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 3, 2022, 7:35pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/8 "2022-03-03T19:35:17Z")

</div>

`baseCommand` only accepts static strings, once you have a dynamic element you have to start using `arguments` for it and all remaining elements of the command line (including other static elements). Likewise `valueFrom` is valid as part of an `arguments` element, but not in `baseCommand`.

```auto
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool

requirements:
  InitialWorkDirRequirement:
    listing:
      - $(inputs.src)

inputs:
  src:
    type: File
    inputBinding:
      position: 1
      valueFrom: $(self.basename)

outputs:
  classfile: stdout

baseCommand: echo

arguments:
  - $(runtime.outdir)

```

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 4, 2022, 6:36am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/9 "2022-03-04T06:36:39Z")

</div>

My workflow consists of multiple iterations and as loops are not implemented in CWL yet. I am writing the workflow for a single iteration and creating the parameter file at the end of each iteration which will be used by the next iteration again.

And for every iteration, I have to check the files from the Results directory, if it exists already read and append data to the current iteration file, if not create an empty file (to avoid the File not found error).

And that was the reason I was not staging files but accessing them through an absolute path. And for the very same reason, I wanted to define the ENV\_VAR at the workflow level, so I don’t have to pass the same arguments most of the time and it might be easier for users to run it without much knowledge of CWL.

I am not putting the same code for the number of iterations I want to run, which was the solution I found in one of the posts because I want it to be dynamic and for different use cases the number of iterations are different.

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 10, 2022, 8:59am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/10 "2022-03-10T08:59:53Z")

</div>

Thanks for the context 👍

Using path strings instead of File or Directory inputs/outputs will eventually cause you a big problem as it violates a core element of the CWL data model. Distributed execution, using caching, provenance recording, and other scenarios will likely break.

Maybe you can modify your loop handling logic (outside the CWL workflow) to merge results and produce the inputs for the next iteration instead of doing that at the end of your CWL workflow?

Then your CWL workflow can use proper File and/or Directory inputs and outputs.

We are [actively discussing](https://github.com/common-workflow-language/common-workflow-language/issues/495#issuecomment-1063807411) how to design a `loop` feature for CWL, so if you can share more details about your situation then that would be very helpful. A complete copy of your workflow and the loop logic with example inputs and expected outputs would be the best; but if you can’t share the data or code then maybe you can describe it enough thay we can build a mock version together.

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 10, 2022, 4:04pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/11 "2022-03-10T16:04:52Z")

</div>

For the moment, I can not share many details; but here’s the brief overview of the workflow I’m trying to build:

I create several files in the first iteration of the loop using variables(string as a file name) specified by a user (considering the user might want to change the names accordingly)

And some of these files I need in the next iterations to keep track of everything. So now I have to give the filenames as strings for the first iteration and File for the next iterations.

I am not sure whether we can check if a file already exists using CWL and create an empty file if it does not exist. If this feature is already there, then it would most probably solve my problem of working around with filenames as strings.  
Another workaround is to simply stage the whole result directory but that would not be a wise decision because file size is quite large.

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 10, 2022, 6:18pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/12 "2022-03-10T18:18:22Z")

</div>

Thanks for the information @HrishiDhondge

Here is an example of taking a `File` or a creating a file from `string`

```nohighlight
cwlVersion: v1.0
class: CommandLineTool

inputs:
 opt_file:
  type: [File, string]

requirements:
 InlineJavascriptRequirement: {}
 InitialWorkDirRequirement:
   listing: |
     ${
       if (typeof inputs.opt_file === 'string') {
         return [{"class": "File", "basename": inputs.opt_file, "contents": "" }]; }
       else { return [inputs.opt_file] ; }
      }

baseCommand: [ls, -l]

arguments: [$(inputs.opt_file)]

outputs: []

```

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 11, 2022, 7:11am UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/13 "2022-03-11T07:11:42Z")

</div>

Thank you @mrc  
I am almost in the final stage of the workflow, so I’ll try to finish it and hopefully, I will be able to share the code and everything to build a test case for loops. 🙂

How may I implement the [default value](https://cwl.discourse.group/t/constructing-a-default-value-for-a-tool-input/161/4) from this example for File? I am trying to access the file from it’s location and read it inside the tool?

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [March 12, 2022, 3:34pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/14 "2022-03-12T15:34:56Z")

</div>

> [@HrishiDhondge](#):
>
> How may I implement the [default value](https://cwl.discourse.group/t/constructing-a-default-value-for-a-tool-input/161/4) from this example for File? I am trying to access the file from it’s location and read it inside the tool?

To add a default value to `opt_file`, try the example below using your desired default file name instead of `my_default_file_name.txt`

```auto
inputs:
 opt_file:
  type: [File, string, "null"]
  default: my_default_file_name.txt

```

Or you can provide a default value at the workflow or workflow step level, which will flow to this `CommandLineTool`. Your choice!

To get the path of the file, use `$(inputs.opt_file)` like in my example

---

<div class="post-metadata">

**Author:** ![HrishiDhondge](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/hrishidhondge/32/276_2.png) [@HrishiDhondge](https://cwl.discourse.group/u/HrishiDhondge)\
**Post date:** [March 12, 2022, 8:34pm UTC](https://cwl.discourse.group/t/access-envdef-of-workflow-in-cwltool/572/15 "2022-03-12T20:34:56Z")

</div>

> [@mrc](#):
>
> ```auto
> InitialWorkDirRequirement:
> listing: |
> ${
> if (typeof inputs.opt_file === 'string') {
> return [{"class": "File", "basename": inputs.opt_file, "contents": "" }]; }
> else { return [inputs.opt_file] ; }
> }
> 
> ```

Just an additional comment would be add `writable= true` in if statement to create a file from string with read, write access. Without the writable option, it create files with read access only.

```auto
InitialWorkDirRequirement:
   listing: |
     ${
       if (typeof inputs.opt_file === 'string') {
         return [{"class": "File", "basename": inputs.opt_file, "contents": "" , writable: true}]; }
       else { return [inputs.opt_file] ; }
      }

```