Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/migrations/dsl1.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,14 @@ cheers

Similarly, in DSL2, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.

## Modules
## Includes

In DSL1, the entire Nextflow pipeline must be defined in a single file. For example, `main.nf`. This restriction becomes cumbersome as a pipeline grows and hinders the sharing and reuse of pipeline components.

DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2, modules can help you organize a large pipeline into multiple smaller files and take advantage of modules created by others. See {ref}`modules-page` to learn more about modules.
DSL2 introduces the ability for scripts to *include* other scripts. While splitting a pipeline into multiple scripts is not mandatory in DSL2, it is useful for organizing a large pipeline and re-using modules created by others. See {ref}`modules-page` for more information.

:::{note}
DSL2 scripts cannot exceed 64 KB in size. Split large DSL1 scripts into modules to avoid this limit.
DSL2 scripts cannot exceed 64 KB in size. Split large DSL1 scripts into multiple smaller scripts to avoid this limit.
:::

## Deprecations
Expand Down Expand Up @@ -176,7 +176,7 @@ An early preview of DSL2 was available in 2020. Note that some of that early DSL

- The `nextflow.preview.dsl=2` (and `nextflow.enable.dsl=1`) feature flags are no longer needed.

- Anonymous and unwrapped includes are no longer supported. Use an explicit module inclusion instead.
- Anonymous and unwrapped includes are no longer supported. Include each definition by name instead.

For example:

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/developing-modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Registry modules are subject to the following constraints:
- The module may define an entry workflow to override the default behavior of `module run`.
- The module may define any number of functions

Local modules can define any number of processes, workflows, and functions. As a best practice, each process and named workflow should be defined in its own script.
Local scripts can define any number of processes, workflows, and functions. As a best practice, each process and named workflow should be defined in its own script.

### meta.yml

Expand Down
6 changes: 3 additions & 3 deletions docs/modules/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
# Overview

Nextflow scripts can include **definitions** (workflows, processes, and functions) from other scripts.
When a script is included in this way, it is referred to as a **module**.
A **module** is a self-contained process definition and corresponding {ref}`spec file <dev-modules-structure>`.
Modules can be re-used within a pipeline and can be shared across projects.
By packaging definitions as modules, you avoid duplicating code and benefit from community improvements.
By packaging process definitions as modules, you avoid duplicating code and benefit from community improvements.

There are two ways to use modules in Nextflow:

Expand All @@ -29,7 +29,7 @@ workflow {
}
```

The above snippet imports a process named `CAT` from the *included module* into the *including script*. The include source `./modules/cat` refers to the script `./modules/cat.nf` relative to the including script path.
The above snippet imports a process named `CAT` from the *included script* into the *including script*. The include source `./modules/cat` refers to the script `./modules/cat.nf` relative to the including script path.

Local modules are well suited for project-specific components that are not intended for sharing.

Expand Down
8 changes: 4 additions & 4 deletions docs/reference/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ workflow {

In order for a script to be executable, it must either define an entry workflow or be a code snippet as described [above](#script-declarations).

Entry workflow definitions are ignored when a script is included as a module. This way, the same script can be included as a module or executed as a pipeline.
Entry workflow definitions are ignored when a script is included by another script. This way, the same script can be executed as a pipeline or included in another pipeline.

(syntax-workflow-typed)=

Expand Down Expand Up @@ -249,7 +249,7 @@ process greet {
errorStrategy 'retry'
tag { "${greeting}/${name}" }

input:
input:
val greeting
val name

Expand Down Expand Up @@ -280,7 +280,7 @@ The script section can be substituted with an exec section:

```nextflow
process greet {
input:
input:
val greeting
val name

Expand All @@ -306,7 +306,7 @@ A typed process is a process that uses static typing for inputs and outputs:
nextflow.enable.types = true

process greet {
input:
input:
greeting: String
name: String

Expand Down
33 changes: 28 additions & 5 deletions docs/script.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Nextflow uses UTF-8 as the default character encoding for source files. Make sur
:::

:::{warning}
Nextflow scripts have a maximum size of 64 KiB. To avoid this limit for large pipelines, consider moving pipeline components into separate files and including them as modules.
Nextflow scripts have a maximum size of 64 KiB. To avoid this limit for large pipelines, consider moving pipeline components into separate files and including them in the main script.
:::

## Hello world
Expand Down Expand Up @@ -129,7 +129,7 @@ age = person.age
is_alive = person.is_alive
```

Records are immutable -- once a record is created, it cannot be modified. Use record operations to create new records instead.
Records are immutable -- once a record is created, it cannot be modified. Use record operations to create new records instead.

For example:

Expand Down Expand Up @@ -532,7 +532,7 @@ When a closure takes a single parameter, the parameter can be omitted, in which

So far, we have been focusing on the basic building blocks of Nextflow code, like variables, lists, strings, and closures.

In practice, however, Nextflow scripts are composed of *workflows*, *processes*, and *functions* (collectively known as *definitions*), and can *include* definitions from other scripts.
In practice, however, Nextflow scripts are composed of *workflows*, *processes*, and *functions* (collectively known as *definitions*), and can *include* definitions from other scripts.

To transition a code snippet into a proper workflow script, simply wrap it in a `workflow` block:

Expand All @@ -544,7 +544,7 @@ workflow {

This block is called the *entry workflow*. It serves as the entrypoint when the script is executed. A script can only have one entry workflow. Whenever a script contains only simple statements like `println 'Hello!'`, Nextflow simply treats it as an entry workflow.

You can also break up code into functions, for example:
You can break up code into functions, for example:

```nextflow
def sayHello() {
Expand All @@ -561,4 +561,27 @@ workflow {
}
```

See {ref}`Workflows <workflow-page>`, {ref}`Processes <process-page>`, and {ref}`Modules <modules-page>` for more information about how to use these features in your Nextflow scripts.
You can organize these definitions further by moving them to a separate script and including them in the main script:

```nextflow
// functions.nf
def sayHello() {
println 'Hello!'
}

def add(a, b) {
a + b
}
```

```nextflow
// main.nf
include { sayHello; add } from './functions.nf'

workflow {
sayHello()
println "2 + 2 = ${add(2, 2)}!"
}
```

See {ref}`Workflows <workflow-page>`, {ref}`Processes <process-page>`, and {ref}`Includes <syntax-include>` for more information about how to use these features in your Nextflow scripts.
4 changes: 2 additions & 2 deletions docs/strict-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ workflow {
```

:::{note}
Mixing statements and script declarations was necessary in DSL1 and optional in DSL2. However, this pattern is not supported by the strict parser in order to ensure that top-level statements are not executed when the script is included as a module.
Mixing statements and script declarations was necessary in DSL1 and optional in DSL2. However, this pattern is not supported by the strict parser in order to ensure that top-level statements are not executed when the script is included by another script.
:::

### Assignment expressions
Expand Down Expand Up @@ -729,4 +729,4 @@ There are two ways to preserve Groovy code:

Any Groovy code can be moved into the `lib` directory, which supports the full Groovy language. This approach is useful for temporarily preserving some Groovy code until it can be updated later and incorporated into a Nextflow script. See {ref}`lib-directory` documentation for more information.

For Groovy code that is complicated or if it depends on third-party libraries, it may be better to create a plugin. Plugins can define custom functions that can be included by Nextflow scripts like a module. Furthermore, plugins can be easily re-used across different pipelines. See {ref}`dev-plugins-page` for more information on how to develop plugins.
For Groovy code that is complicated or if it depends on third-party libraries, it may be better to create a plugin. Plugins can define custom functions that can be included by Nextflow scripts like a script definition. Furthermore, plugins can be easily re-used across different pipelines. See {ref}`dev-plugins-page` for more information on how to develop plugins.
Loading