Making an input type: File writable

The following cwl is valid, completed with “Final process status is success”, however the input collected_statistics_md (underlying file) does not have the new data line concatenated as expected by the bash script (at line 35).

cwltool version 3.1.20210816212154

cwlVersion: v1.0
class: CommandLineTool


requirements:
- class: ShellCommandRequirement
- class: InlineJavascriptRequirement
- class: InitialWorkDirRequirement
  listing:
    - entry: $(inputs.collected_statistics_md)
      writable: true


hints:
- class: DockerRequirement
  dockerPull: biowardrobe2/samtools:v1.4


inputs:

  script:
    type: string?
    default: |
      #!/bin/bash
      printf "$(date)\nLog file for calc_frip function in collect-statistics-frip.cwl tool:\n\n" > "calc_frip.log"
      bam="$0"; bed="$1"; md="$2"; tsv="$3"; yaml="$4";
      # count of total aligned reads
      tar=$(samtools view -cF0x4 "$bam" 2>> "calc_frip.log")
      # counts of reads in peak
      rip=$(samtools view -c "$bam" -L "$bed" 2>> "calc_frip.log")
      # frip=rip/tar
      frip=$(printf "$tar" | awk -v rip="$rip" '{printf("%.3f",rip/$0)}' 2>> "calc_frip.log")
      printf "$tar, $rip, $frip\n" >> "calc_frip.log"
      # concatenate frip onto md, tsv, and yaml files
      printf "fraction of (aligned) reads in peaks: $frip\n" >> $md 2>> "calc_frip.log"
#      printf "fraction of (aligned) reads in peaks: $frip\n" >> "$md"
#      printf "fraction of (aligned) reads in peaks: $frip\n" >> "$tsv"
#      printf "fraction of (aligned) reads in peaks: $frip\n" >> "$yaml"
    inputBinding:
        position: 4

  bam_file:
    type: File
    label: "Input BAM file"
    inputBinding:
      position: 5
    doc: "Input BAM file, does not have to be coordinates sorted"

  seacr_called_peaks:
    type: File
    inputBinding:
      position: 11

  collected_statistics_md:
    type: File
    inputBinding:
      position: 20

  collected_statistics_tsv:
    type: File
    inputBinding:
      position: 21

  collected_statistics_yaml:
    type: File
    inputBinding:
      position: 22


outputs:

  log_file:
    type: File
    outputBinding:
      glob: "calc_frip.log"


baseCommand: ["bash", "-c"]

You need to collect the modified input file as one of your outputs (the real original input file does not get modified unless you use InplaceUpdateRequirement which is not generally recommended.

Here is a proof of concept:

cwlVersion: v1.1
class: CommandLineTool

inputs:
  starting_file: File

requirements:                                                                   
  InitialWorkDirRequirement: 
    listing:
      - entry: $(inputs.starting_file)
        writable: True

baseCommand: bash

arguments:
 - -c
 - "echo extra >> $(inputs.starting_file.path)"

outputs:
 modified_file:
   type: File
   outputBinding:
    glob: $(inputs.starting_file.basename)
1 Like

Perfect, works as expected and really appreciate the explanation with example!

1 Like

Ah wait… I think I fooled myself into thinking it was working.

I do not have my bash script wrapped exactly like you do in your example, is that a potential issue? Anything else I could try? Right now the input files I want to modify are being written to the new output, but there is nothing being concatenated as expected in the bash script and directs the data stream.

I don’t know if you can combine a bash -c invokation with $1 bash references.

So either use CWL parameter references instead of inputBinding: { position: N } like I did, or create your script using https://www.commonwl.org/user_guide/topics/creating-files-at-runtime.html and then reference the path to your script

Oh I see, I will keep that in mind going forward. I was able to find a workable solution to my particular issue, and thank you again for the help here!

1 Like