Good day,
I am building a transpiler that reads CWL processes and generates Python code that runs the tools/workflows using DASK. I am reading the CWL process files with cwl_utils.parser.load_document_by_uri, which works fine until encountering an $include statement in InlineJavascriptRequirement.expressionLib. I know that this happens because workflow engines that implement CWL are expected to replace the $include statements with their targets during preprocessing. I am struggling to find a way around this without pivoting away from using loader altogether, which I really do not want to do. Any ideas or solutions?
Welcome @kennethdirker and thank you for sharing your issue.
What version of cwl-utils are you using? Can you share a minimal example that reproduces the error?
I was not able to reproduce your issue, alas.
I applied the following diff to load tool1.cwl and dump its output. It uses an $include inside a InlineJavascriptRequirement.
$ git diff
diff --git a/src/cwl_utils/tests/load_cwl_by_path.py b/src/cwl_utils/tests/load_cwl_by_path.py
index 82e989a..bc07173 100644
--- a/src/cwl_utils/tests/load_cwl_by_path.py
+++ b/src/cwl_utils/tests/load_cwl_by_path.py
@@ -10,7 +10,7 @@ from pathlib import Path
from cwl_utils.parser import load_document_by_uri, save
# File Input - This is the only thing you will need to adjust or take in as an input to your function:
-cwl_file = Path("../testdata/md5sum.cwl") # or a plain string works as well
+cwl_file = Path("src/cwl_utils/testdata/remote-cwl/tool1.cwl") # or a plain string works as well
# Import CWL Object
cwl_obj = load_document_by_uri(cwl_file)
the result does show that the javascript file was included
$ python load_cwl_by_path.py
Export of the loaded CWL object: {'id': 'file:///home/michael/cwl-utils/src/cwl_utils/testdata/remote-cwl/tool1.cwl', 'class': 'CommandLineTool', 'inputs': [{'id': 'in1', 'type': 'string', 'inputBinding': {'position': 1, 'valueFrom': 'A_$(inputs.in1)_B_${return inputs.in1}_C_$(inputs.in1)'}}], 'outputs': [{'id': 'out1', 'type': 'string', 'outputBinding': {'loadContents': True, 'glob': 'out.txt', 'outputEval': '$(self)_D_$(runtime)'}}], 'requirements': [{'class': 'InlineJavascriptRequirement', 'expressionLib': ['var foo = function(x) {\n return 2 * x\n}\n\nvar bar = function(n, x) {\n return n + " engineers walk into a " + x\n}\n']}], 'cwlVersion': 'v1.2', 'baseCommand': 'echo', 'arguments': [{'valueFrom': '$(runtime)'}], 'stdout': 'out.txt'}.
I even tried changing the requirements to hints (though that wouldn’t make much sense), and the $include still worked.