Skip to content

Record Path fields are not staged when the record type is included from another module #7225

@robsyme

Description

@robsyme

Summary

With the typed/static-types syntax (nextflow.enable.types = true), a process input that is a Path field of a record type included from another module file (e.g. include { Pair } from './types.nf') is not staged. The implicit stageAs directives that should be generated for the record's Path fields are missing, so the field value is interpolated into .command.sh as a raw source path instead of a staged TaskPath.

  • On a local filesystem the bug is masked: the raw absolute path still resolves, so the task succeeds (but rec.r1.getClass() reveals sun.nio.fs.UnixPath instead of nextflow.processor.TaskPath).
  • On an object-store work dir (AWS Batch + Fusion/S3) the raw path is /bucket/key, which never passes through the Fusion mount mapping, and the task fails with No such file or directory.

Defining the same record inline in the script that declares the process stages the fields correctly. Direct Path inputs are also unaffected.

Reproducer

https://github.com/robsyme/nf-record-fusion-repro

export NXF_SYNTAX_PARSER=v2
nextflow run robsyme/nf-record-fusion-repro

MAKE emits Pair(id, r1: Path, r2: Path) where Pair is included from types.nf. CONSUME_A reads rec.r1/rec.r2 (record fields) and fails on Fusion/S3; VIA_PATH reads the same files as direct Path inputs and succeeds.

Isolating the variables:

Record definition Field staged?
included from types.nf no — raw source path
inline in main.nf yes — TaskPath

Evidence

getClass() evaluated at script-render time, same run:

CONSUME_A s02: r1class=class nextflow.cloud.aws.nio.S3Path  r1=/bucket/.../s02_1.txt   # record field -> raw, UNSTAGED
VIA_PATH  s02: r1class=class nextflow.processor.TaskPath     r1=s02_1.txt               # direct Path -> staged

Failed CONSUME_A .command.sh:

cat /bucket/.../s02_1.txt /bucket/.../s02_2.txt > s02.a.out
# -> cat: /bucket/.../s02_1.txt: No such file or directory

Root cause

The v2 script compiler (ScriptCompiler) resolves and converts each source file in a single per-source pass, and the entry script is analyzed before the modules it includes. When ProcessToGroovyVisitorV2.visitProcessInputType inspects a record-typed input to generate the implicit stageAs directives, the included record node is found (isRecordType is true) but its field types are still unresolved ClassNodes — isPathType returns false, so no stager is emitted for the Path fields.

At runtime the unstaged field then misses the staged-path substitution in TaskInputResolver.normalizeValue (the path is absent from the staged-file holders map) and reaches the task script as a raw source path.

Environment

  • Nextflow 26.04.2 / 26.04.3, also reproduced on master; strict v2 parser (NXF_SYNTAX_PARSER=v2)
  • Any executor (staging bug visible locally); failure surfaces on AWS Batch + Fusion v2 + Wave

Edited: the original report attributed the failure to nextflow.enable.moduleBinaries. That was a misdiagnosis — the flag is unrelated; the trigger is the record type being included from a separate module file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions