Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,67 @@ jobs:
path: target/ncbi-policy.json
if-no-files-found: error

reference-workflows:
name: Reference workflows
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read

steps:
- name: Checkout
uses: actions/checkout@v6.0.2

- name: Install Rust
uses: dtolnay/rust-toolchain@stable

- name: Install Python
uses: actions/setup-python@v6
with:
python-version: "3.12"

- name: Cache Cargo
uses: Swatinem/rust-cache@v2

- name: Install Nextflow
uses: nf-core/setup-nextflow@893c28b667aedeba26e37f296d260ccc5bc4d914 # v3.0.1
with:
version: "26.04.6"
java-version: "17"

- name: Install Snakemake and test dependencies
run: python3 -m pip install --requirement requirements-test.txt "snakemake==9.25.2"

- name: Build FastaGuard
run: cargo build --locked --bin fastaguard

- name: Check workflow engine versions
run: |
nextflow -version
snakemake --version

- name: Lint reference workflows
run: |
nextflow lint examples/reference/nextflow/main.nf
snakemake --snakefile examples/reference/snakemake/Snakefile \
--configfile examples/reference/snakemake/config.json \
--config binary="$PWD/target/debug/fastaguard" --lint

- name: Test reference workflows with both engines required
env:
FASTAGUARD_REQUIRE_WORKFLOW_ENGINES: "1"
run: |
python3 -m pytest -q -ra tests/python/test_reference_workflows.py \
--junitxml=target/reference-workflow-tests.xml

- name: Upload workflow test results
if: always()
uses: actions/upload-artifact@v4.6.2
with:
name: reference-workflow-tests
path: target/reference-workflow-tests.xml
if-no-files-found: warn

multiqc-plugin:
name: MultiQC plugin (Python ${{ matrix.python }}, MultiQC ${{ matrix.multiqc }})
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,7 @@ target/
.worktrees/
__pycache__/
*.py[cod]
/.nextflow/
/.nextflow.log*
/.snakemake/
/work/
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ cff-version: "1.2.0"
message: "If you use FastaGuard, please cite this software."
type: software
title: "FastaGuard"
version: "1.0.0"
version: "1.1.0"
repository-code: "https://github.com/ehsanestaji/FastaGuard"
license: "MIT"
authors:
Expand Down
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "fastaguard"
version = "1.0.0"
version = "1.1.0"
edition = "2021"
license = "MIT"
description = "FASTA preflight QC for assembly pipelines"
Expand Down
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,16 @@ downstream tool.

| Channel | Status |
| --- | --- |
| Development source | `v1.1.0` (unreleased) |
| Source release | `v1.0.0` Reference Contract Gate |
| GitHub release | `v1.0.0` release binaries and checksums |
| Bioconda | `v1.0.0` is live for `linux-64`, `linux-aarch64`, `osx-64`, and `osx-arm64` |
| BioContainers | `1.0.0--hfa8f182_0` is the published pinned workflow image |
| Source build | local checkout builds report the package version from `Cargo.toml` |

The [v1.1.0 release notes](docs/releases/v1.1.0.md) describe the reference
workflow examples and CI checks being prepared for the next release.

## Install

For the shortest published-version path from installation to four local
Expand Down Expand Up @@ -291,7 +295,9 @@ not accepted as an annotation declaration.

Try the [reference quickstart](docs/quickstart.md#reference-quickstart) for a
matching reference and a deliberately mismatched dictionary, with retained
reports and a separate policy gate.
reports and a separate policy gate. The
[reference workflow examples](examples/reference/README.md) run the same
collect-then-gate pattern in Nextflow and Snakemake.

Reference mode writes HTML, JSON, TSV and MultiQC-compatible JSON by default.
Use `--format` to select a subset, `--write-lock` to write a
Expand Down Expand Up @@ -501,7 +507,7 @@ FastaGuard catches FASTA-level assembly problems before expensive assembly QC.
- [v0.1.0 release notes](docs/releases/v0.1.0.md)
- [Roadmap](docs/roadmap.md)

Citation metadata for the v1.0.0 source release is provided in
Citation metadata for this source version is provided in
[`CITATION.cff`](CITATION.cff).

## Status
Expand Down
13 changes: 11 additions & 2 deletions docs/packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ The current published package includes the `--gate submission` and
`--submission-target generic|ncbi` contract. Keep future docs pinned to
confirmed Bioconda and BioContainers versions before advertising them as live.

The development source targets v1.1.0, which is not yet released. Its
[release notes](releases/v1.1.0.md) describe the planned workflow additions.
Keep the recipe, source archive checksum and workflow image pins at the
published version until the new source archive and packages are available.

## Bioconda

Recommended install:
Expand Down Expand Up @@ -106,8 +111,12 @@ Build and inspect a host archive locally:

```bash
cargo build --release --locked
scripts/package_release_artifact.sh "$(rustc -vV | sed -n 's/^host: //p')" v1.0.0
tar -tzf "dist/fastaguard-v1.0.0-$(rustc -vV | sed -n 's/^host: //p').tar.gz"
package_id="$(cargo pkgid --quiet)"
release_version="${package_id##*@}"
host_target="$(rustc -vV | sed -n 's/^host: //p')"
scripts/check_release_tag.sh "v${release_version}"
scripts/package_release_artifact.sh "$host_target" "v${release_version}"
tar -tzf "dist/fastaguard-v${release_version}-${host_target}.tar.gz"
```

Each archive has one top-level directory containing `fastaguard`, `README.md`,
Expand Down
3 changes: 3 additions & 0 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ It is not a reusable workflow gate. Use a new output directory to repeat the
example while retaining previous evidence; existing files are not overwritten.
The first argument can be a quoted executable path instead of `fastaguard`.

For complete Nextflow and Snakemake examples, see the
[reference workflows](../examples/reference/README.md).

### Write a reference lockfile

`--write-lock` is a boolean switch. With `--outdir`, the lockfile is named
Expand Down
58 changes: 58 additions & 0 deletions docs/releases/v1.1.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# FastaGuard v1.1.0 - Reference Workflow Adoption

Status: unreleased. This source tree targets v1.1.0; published downloads remain
at v1.0.0.

## Reference workflows

This release adds Nextflow and Snakemake examples for using
`fastaguard reference` before downstream work. Both support a single reference
bundle on a local or shared filesystem, with explicit FASTA and companion-file
dependencies.

- `report` mode saves the reports without enforcing the compatibility decision.
- `enforce` mode checks `gate.can_continue` before a demonstration downstream
task can run.
- A blocked gate leaves the completed HTML, JSON, TSV and MultiQC reports
available for inspection. A reference lockfile is optional.
- Input and report checks protect existing evidence during retry or resume;
changed inputs require a new run identifier.

The [workflow examples](../../examples/reference/README.md) include the
configuration files and synthetic fixtures. They are supplied in the repository
source, not the standalone binary archives. They do not modify input files or
perform a biological downstream analysis. Object-store publication and
multi-bundle orchestration are outside their scope.

## Verification and documentation

The dedicated CI job pins Nextflow 26.04.6 with Java 17 and Snakemake 9.25.2
with Python 3.12. It runs both workflow linters, the eight-case decision matrix,
report retention and retry checks. Required mode fails if an engine is missing
or a fixture filter would reduce the matrix.

The reference quickstart and installation guidance now separate source builds
from published packages. Reference memory use, identity guarantees and
publication limits are documented more precisely. The workflow fixtures test
software behaviour; they are not additional biological validation evidence.

## Compatibility

There are no changes to the CLI's analysis or exit-code contracts. Assembly and
compare reports retain schema version `0.7.0`. Reference reports retain schema
version `1.0.0`. The new workflow examples also work with the published v1.0.0
reference command.

The existing nf-core module and Snakemake wrapper are unchanged. These local
reference examples are not new upstream workflow components.

## Distribution

Bioconda and BioContainers remain at v1.0.0. The recipe checksum, container tags
and upstream wrapper pins remain tied to that published version. They will be
updated only after the v1.1.0 source archive and downstream packages are
available.

Before publication, the release commit must pass hosted CI and the
cross-platform release builds. The release archives and checksum manifest must
then be verified. Preparing this source tree does not publish a release.
14 changes: 14 additions & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,20 @@ GitHub, Bioconda, and BioContainers. Bioconda serves `linux-64`,
`linux-aarch64`, `osx-64`, and
`osx-arm64`; the published BioContainers tag is `1.0.0--hfa8f182_0`.

## v1.1: Reference Workflow Adoption

Status: in preparation, not yet released.

- tested Nextflow and Snakemake examples for a single reference bundle
- separate reporting and enforcement, with reports retained when a gate blocks
- explicit input dependencies and checks for changed inputs or reports on retry
- pinned workflow-engine CI that requires the full decision matrix
- clearer installation, reference quickstart and compatibility guidance

This release keeps the v1.0 command-line behaviour and report schemas.
It does not introduce new upstream nf-core or Snakemake reference components.
See the [v1.1.0 release notes](releases/v1.1.0.md) for scope and limitations.

## v1.0: Reference Contract Gate

Goal:
Expand Down
6 changes: 4 additions & 2 deletions docs/workflow-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,10 @@ starters remain local compatibility references.

Reference-specific workflow components are not yet available upstream.
Use the [reference quickstart](quickstart.md#reference-quickstart) to exercise
the published CLI directly; a version update to an assembly wrapper does not
add reference inputs to its interface.
the published CLI directly, or the local
[Nextflow and Snakemake reference workflows](../examples/reference/README.md)
for report retention and downstream gating. A version update to an assembly
wrapper does not add reference inputs to its interface.

Historical v0.5 upstream-style validation was run on 2026-07-03 in dedicated
checkouts:
Expand Down
88 changes: 88 additions & 0 deletions examples/reference/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Reference workflows

These single-bundle examples run the published FastaGuard v1.0.0 reference
command. Run them from the repository root with FastaGuard and the chosen
workflow engine on `PATH`. Each example uses the same synthetic reference and
dictionary as the [reference quickstart](../../docs/quickstart.md#reference-quickstart).

Nextflow (DSL2):

```bash
nextflow run examples/reference/nextflow/main.nf \
-params-file examples/reference/nextflow/params.json
```

Snakemake:

```bash
snakemake --snakefile examples/reference/snakemake/Snakefile --cores 1 \
--configfile examples/reference/snakemake/config.json
```

The two checked-in configurations use different output roots. Each writes a
run directory named `exact` containing HTML, JSON, TSV, MultiQC JSON, a
reference lockfile, and `downstream.started` after the gate accepts the bundle.
The lockfile is optional and has its own output declaration. The marker is a
demonstration of a downstream task, not a biological analysis.

Use a native JSON parameter/configuration file for a real bundle. Both engines
accept these keys:

| Key | Meaning |
| --- | --- |
| `fasta` | Required canonical FASTA path. |
| `output_dir` | Required local or shared-filesystem output root. |
| `run_id` | Unique name for this input set; letters, digits, `_` and `-`, starting with a letter or digit. Defaults to `reference`. |
| `mode` | `report` (default) or `enforce`. |
| `fai`, `dict`, `alias_map` | Optional explicit file paths. |
| `alignments`, `variants`, `annotations` | Lists of explicit file paths; each item is a workflow file dependency. |
| `policy` | `strict`, `coordinate` (default), or `advisory`. |
| `require` | List drawn from `fai`, `dict`, `alignment`, `variants`, `annotation`. |
| `write_lock` | Boolean; when true, request a lockfile alongside the reports. |
| `binary` | Optional executable path or command; defaults to `fastaguard`. |

An omitted optional file differs from a supplied unreadable path. All four
reports are requested together. Use a new `run_id` when any input or report
setting changes; both examples reject reuse that would contradict an existing
run. Both examples record a hidden `.reference-inputs.sha256` file and hash
the supplied inputs before reporting. Large references add a second streaming
read for that protection. Snakemake also records
`.reference-report-digests.json` and checks the published report files on
each run; missing or modified evidence is an error.

`report` mode stops after saving the reports, including a completed FAIL
report. In `enforce` mode, a separate gate reads the JSON boolean
`gate.can_continue`. A blocked gate fails the workflow while leaving all
completed reports in `output_dir/run_id/`; no downstream marker is written.
The original FASTA and companions are never modified.

Nextflow stages repeated companion filenames in numbered subdirectories. Its
persistence task synchronously copies the bundle to `output_dir/run_id/` and
emits a receipt before the gate can run. It accepts `-resume` for unchanged
inputs only after confirming the published bytes match. Snakemake writes
engine-managed reports to a staging directory, then publishes them without
replacing earlier completed evidence. Its gate rule owns only the acceptance
marker. These examples target one bundle on a local or shared filesystem.
Object-store publication and multiple bundles are outside their scope.

The small [fixture manifest](fixtures/workflow-cases.json) defines eight
expected decision cases used by the engine integration tests. It is test data,
not biological validation evidence.

## Testing

The dedicated GitHub CI job uses Nextflow 26.04.6, Java 17, Snakemake 9.25.2,
and Python 3.12. It runs both workflow linters and the full reference test
file, including the decision matrix, report retention, and retry checks.

With both engines on `PATH` and the Python test dependencies installed:

```bash
cargo build --locked --bin fastaguard
FASTAGUARD_REQUIRE_WORKFLOW_ENGINES=1 \
python3 -m pytest -q -ra tests/python/test_reference_workflows.py
```

Required mode fails when an engine is unavailable and rejects a
`FASTAGUARD_WORKFLOW_CASE` filter. Without required mode, local tests may
skip an unavailable engine.
1 change: 1 addition & 0 deletions examples/reference/fixtures/aliased.fa.fai
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1 4 6 4 5
2 changes: 2 additions & 0 deletions examples/reference/fixtures/aliases.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
declared_name reference_name
1 chr1
3 changes: 3 additions & 0 deletions examples/reference/fixtures/genes.gff3
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
##gff-version 3
##sequence-region chr1 1 4
chr1 FastaGuard gene 1 4 . + . ID=gene1
2 changes: 2 additions & 0 deletions examples/reference/fixtures/header-only.dict
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@HD VN:1.6
@SQ SN:chr1 LN:4
2 changes: 2 additions & 0 deletions examples/reference/fixtures/invalid.fa
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
>chr1
ACGTX
2 changes: 2 additions & 0 deletions examples/reference/fixtures/reads.sam
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@HD VN:1.6
@SQ SN:chr1 LN:4 M5:f1f8f4bf413b16ad135722aa4591043e
1 change: 1 addition & 0 deletions examples/reference/fixtures/reference.fa.fai
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
chr1 4 6 4 5
3 changes: 3 additions & 0 deletions examples/reference/fixtures/variants/a/sites.vcf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
##fileformat=VCFv4.3
##contig=<ID=chr1,length=4>
#CHROM POS ID REF ALT QUAL FILTER INFO
3 changes: 3 additions & 0 deletions examples/reference/fixtures/variants/b/sites.vcf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
##fileformat=VCFv4.3
##contig=<ID=chr1,length=4>
#CHROM POS ID REF ALT QUAL FILTER INFO
Loading
Loading