Information about `cwltool`

CWL Questions
1

I’ve been asked to provide the following information about cwltool for a catalogue of workflow environments are being elaborated the HealthyCloud project. So I thought I’d list my answers publicly, in case I missed something.

  1. Main community Main user community
    Common Workflow Language, a member project of the Software Freedom Conservancy
    cwltool is used to

    1. By workflow language developers to prototype new features for future versions of the CWL standards
    2. By workflow engine developers, as an extensible CWL parsing and execution library
    3. By workflow description authors, for local testing of CWL documents.

  2. URL
    cwltool · PyPI

  3. Open source (Y/N, indicate license)
    Yes, Apache 2.0

  4. Maturity (TRL level or similar)
    TRL-6 (“Technology demonstrated in relevant environment”; cwltool is not for production usage by itself)

  5. Orchestrator level (Platform, Application, Service, Infrastructure)
    Not an orchestrator of external systems.

  6. Main execution paradigm (Task base, MPI, web services …)
    Only for local running of tools, perhaps in parallel, perhaps containerized

  7. Being used with health data (Y/N, which data)
    Yes. See Notable CWL mentions, adoption, and users · common-workflow-language/common-workflow-language Wiki · GitHub

  8. Support for distributed data (Y/N, indicate short summary of how this is achieved)
    Not really. cwltool does support HTTP(S) downloading of remote data.
    Other systems that use the cwltool library do support distributed data, for example: Arvados

  9. Support for execution in multi-domain computing infrastructure (Y/N)
    No. cwltool does not distribute tasks, but other workflow systems based upon cwltool do: toil-cwl-runner, arvados, streamflow, reana

  10. Support for sensitive data (Y/N, how this is dealt with)
    No.

  11. Support for reproducibility/provenance (Y/N, mechanism, i.e. containers, etc)
    Yes. Docker format containers can be run using podman, singularity, udocker, or the docker container engines.
    Provenance reporting is done using common provenance standards in a profile we call CWLProv GitHub - common-workflow-language/cwlprov: Profile for provenance research object of a CWL workflow run

    Farah Zaib Khan, Stian Soiland-Reyes, Richard O Sinnott, Andrew Lonie, Carole Goble, Michael R Crusoe, Sharing interoperable workflow provenance: A review of best practices and their practical application in CWLProv, GigaScience, Volume 8, Issue 11, November 2019, giz095, https://doi.org/10.1093/gigascience/giz095

  12. Main initiatives where it has been used (give number and a short list in case of many)
    As the reference runner for the CWL standards, there are a great many initiatives benefiting from or using cwltool directly: See Notable CWL mentions, adoption, and users · common-workflow-language/common-workflow-language Wiki · GitHub

2 Likes
2

Thanks Michael for the answers. One comment, in question 1 I meant to ask who is the “main user community”. Sorry for the confusion.

1 Like
3

Welcome @Rosa_M_Badia !

No worries, I updated my response to question 1.