Launchers and execution engines¶
AntFlow separates job coordination from workflow execution. The server creates and tracks jobs. A launcher connects those jobs to a computing environment, and a CWL engine orchestrates their processing steps.
Launcher, backend, and engine¶
A launcher fetches eligible jobs and workflow packages, starts execution, reports progress, and publishes results. It can be platform-managed or user-managed. Each launcher uses one backend; multiple launchers can provide different environments for the same server.
The backend connects execution to infrastructure. The engine interprets CWL dependencies and coordinates tool invocations:
| Environment | Engine used by AntFlow |
|---|---|
| A Linux host | cwltool |
| Kubernetes | Calrissian |
| HPC with Slurm | Toil with Singularity containers |
cwltool is the CWL reference implementation and supports validation and
direct local execution. Such a local run does not create an AntFlow job.
Calrissian builds on cwltool to coordinate workflow execution on Kubernetes.
AntFlow uses a Calrissian fork;
the deployed version and configuration determine its behavior.
Runner environment and application containers¶
The environment hosting the CWL engine is distinct from the containers used by processing tools. A workflow is not simply one application container containing all its steps.
| Work | Responsible environment |
|---|---|
| Workflow dependencies, scatter, and conditions | The CWL engine coordinates them. |
| CWL expressions and ExpressionTools | The runner's expression evaluator. |
| Input staging, path mapping, and output collection | The runner and its available storage. |
| A tool's application command | Its selected application container, or a direct subprocess when configured without one. |
For Kubernetes, the Calrissian executor runs separately from the launcher and starts application tool invocations in separate pods. For local execution, the launcher starts the engine as a subprocess, inside the launcher container when deployed that way.
JavaScript needed by CWL expressions belongs to the runner's evaluation
environment. Installing Node.js only in an application's image does not
necessarily provide it to the evaluator. Likewise, creating a script through
InitialWorkDirRequirement does not execute it until an application command
invokes it.
Choosing an environment¶
Containerized execution is per tool invocation. A scatter operation can invoke the same tool several times. Effective CWL requirements and executor defaults determine the application image and container runtime.
Starting a launcher with Podman does not itself select Podman for every application command. Use executor configuration and execution logs to confirm what actually runs. Registry access, available resources, data connectivity, and supported extensions all affect portability.
A workflow's flavor labels filter eligible launchers; they do not allocate GPUs or memory by themselves. See Launcher capabilities.
For the component diagram, see Architecture. For deployment and registration, use the launcher guides. For direct testing, use the cwltool documentation.