# Best Practices for Avoiding "Argument List Too Long" in Large-Scale Workflows

**URL:** <https://cwl.discourse.group/t/best-practices-for-avoiding-argument-list-too-long-in-large-scale-workflows/1008>\
**Category:** CWL Questions\
**Created:** [July 4, 2025, 5:40pm UTC](https://cwl.discourse.group/t/best-practices-for-avoiding-argument-list-too-long-in-large-scale-workflows/1008 "2025-07-04T17:40:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jma1991](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/jma1991/32/600_2.png) [@jma1991](https://cwl.discourse.group/u/jma1991)\
**Post date:** [July 4, 2025, 5:40pm UTC](https://cwl.discourse.group/t/best-practices-for-avoiding-argument-list-too-long-in-large-scale-workflows/1008/1 "2025-07-04T17:40:08Z")

</div>

Hi everyone,

I’m looking for some clarity on issues I frequently encounter when working with large-scale processing workflows. Here’s the usual scenario:

I have a tool that processes a single file, and it is scattered across an array of thousands of input files. A second tool then performs a joint analysis, taking the complete array of output files from the scattered step as input.

I’m running into two main problems:

1. **When the tool doesn’t accept a manifest file:**  
How can I avoid the “argument list too long” error when passing all the file paths directly to the tool (e.g., `tool_name file1 file2 ...` with thousands of files)?
2. **When the tool _does_ accept a manifest file:**  
Even then, I sometimes hit the “argument list too long” error at the Docker level due to having thousands of `-v` bind mounts in the command.

Are there recommended strategies or workarounds for these scenarios? Any advice or best practices would be much appreciated.

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![mrc](https://yyz2.discourse-cdn.com/free1/user_avatar/cwl.discourse.group/mrc/32/3_2.png) [@mrc](https://cwl.discourse.group/u/mrc)\
**Post date:** [July 7, 2025, 12:48pm UTC](https://cwl.discourse.group/t/best-practices-for-avoiding-argument-list-too-long-in-large-scale-workflows/1008/2 "2025-07-07T12:48:44Z")

</div>

Hello @jma1991 and welcome! Which CWL runner are you using?