ONT and Illumina Variant Processing Cleanup

[DarianHole]

Exposing new parameters and cleaning up some code for the illumina and nanopore variant processing.

No differences in final consensus output in the test dataset (200 samples) noted. Will run a few more checks as well

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• Make sure your code lints (nf-core pipelines lint).
• Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
• Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
• Usage Documentation in docs/usage.md is updated.
• Output Documentation in docs/output.md is updated.
• CHANGELOG.md is updated.
• README.md is updated (including new tool citations and authors/contributors).

[github-actions]

This PR is against the main branch ❌

• Do not close this PR
• Click Edit and change the base to dev
• This CI test will remain failed until you push a new commit

---

Hi @DarianHole,

It looks like this pull-request is has been made against the phac-nml/measeq main branch.
The main branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to main are only allowed if they come from the phac-nml/measeq dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

[github-actions]

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 4246272

+| ✅ 180 tests passed       |+
#| ❔  52 tests were ignored |#
!| ❗   1 tests had warnings |!

Details

❗ Test warnings:

• nextflow_config - Config manifest.version should end in dev: 1.3.0

❔ Tests ignored:

• files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File is ignored: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File is ignored: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File is ignored: .github/CONTRIBUTING.md
• files_exist - File is ignored: .github/.dockstore.yml
• files_exist - File is ignored: conf/igenomes.config
• files_exist - File is ignored: conf/igenomes_ignored.config
• files_exist - File is ignored: conf/test.config
• files_exist - File is ignored: conf/test_full.config
• files_exist - File is ignored: assets/email_template.txt
• files_exist - File is ignored: assets/email_template.html
• files_exist - File is ignored: assets/sendmail_template.txt
• files_exist - File is ignored: assets/adaptivecard.json
• files_exist - File is ignored: assets/slackreport.json
• files_exist - File is ignored: assets/nf-core-measeq_logo_light.png
• files_exist - File is ignored: docs/images/nf-core-measeq_logo_light.png
• files_exist - File is ignored: docs/images/nf-core-measeq_logo_dark.png
• files_exist - File is ignored: .github/workflows/awstest.yml
• files_exist - File is ignored: .github/workflows/awsfulltest.yml
• files_exist - File is ignored: assets/multiqc_config.yml
• files_exist - File is ignored: CODE_OF_CONDUCT.md
• files_exist - File is ignored: ro-crate-metadata.json
• nextflow_config - Config variable ignored: manifest.name
• nextflow_config - Config variable ignored: manifest.homePage
• nextflow_config - Config variable ignored: process.cpus
• nextflow_config - Config variable ignored: process.memory
• nextflow_config - Config variable ignored: process.time
• nextflow_config - Config variable ignored: validation.help.beforeText
• nextflow_config - Config variable ignored: validation.help.afterText
• nextflow_config - Config variable ignored: validation.summary.beforeText
• nextflow_config - Config variable ignored: validation.summary.afterText
• files_unchanged - File does not exist: CODE_OF_CONDUCT.md
• files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_unchanged - File does not exist: .github/.dockstore.yml
• files_unchanged - File does not exist: .github/CONTRIBUTING.md
• files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/bug_report.yml
• files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
• files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/feature_request.yml
• files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
• files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
• files_unchanged - File ignored due to lint config: .github/workflows/linting_comment.yml
• files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
• files_unchanged - File does not exist: assets/email_template.html
• files_unchanged - File does not exist: assets/email_template.txt
• files_unchanged - File does not exist: assets/sendmail_template.txt
• files_unchanged - File does not exist: assets/nf-core-measeq_logo_light.png
• files_unchanged - File does not exist: docs/images/nf-core-measeq_logo_light.png
• files_unchanged - File does not exist: docs/images/nf-core-measeq_logo_dark.png
• files_unchanged - File ignored due to lint config: docs/README.md
• files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
• actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/measeq/measeq/.github/workflows/awstest.yml
• multiqc_config - multiqc_config

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: conf/modules.config
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: main.nf
• files_exist - File found: conf/base.config
• files_exist - File found: modules.json
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: docs/images/nf-core-measeq_logo.png
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/NfcoreTemplate.groovy
• files_exist - File not found check: lib/Utils.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/WorkflowMain.groovy
• files_exist - File not found check: lib/WorkflowMeaseq.groovy
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: Singularity
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Found nf-schema plugin
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: validation.help.enabled
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable found: validation.help.command
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config variable (correctly) not found: params.max_cpus
• nextflow_config - Config variable (correctly) not found: params.max_memory
• nextflow_config - Config variable (correctly) not found: params.max_time
• nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
• nextflow_config - Config variable (correctly) not found: params.validationLenientMode
• nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.ivar_trim_min_read_length= 30
• nextflow_config - Config default value correct: params.ivar_offset= 0
• nextflow_config - Config default value correct: params.min_ambiguity_threshold= 0.3
• nextflow_config - Config default value correct: params.max_ambiguity_threshold= 0.75
• nextflow_config - Config default value correct: params.min_indel_threshold= 0.6
• nextflow_config - Config default value correct: params.min_alt_fraction_freebayes= 0.05
• nextflow_config - Config default value correct: params.min_variant_qual_freebayes= 20
• nextflow_config - Config default value correct: params.align_bowtie2= false
• nextflow_config - Config default value correct: params.ont_min_read_length= 200
• nextflow_config - Config default value correct: params.ont_min_base_qual= 12
• nextflow_config - Config default value correct: params.normalise_ont= 500
• nextflow_config - Config default value correct: params.ont_keep_incorrect_primers= false
• nextflow_config - Config default value correct: params.min_variant_qual_c3= 7
• nextflow_config - Config default value correct: params.min_frameshift_qual_c3= 30
• nextflow_config - Config default value correct: params.min_allele_freq_c3= 0.6
• nextflow_config - Config default value correct: params.min_mask_freq_c3= 0.3
• nextflow_config - Config default value correct: params.min_site_threshold_c3= 0.05
• nextflow_config - Config default value correct: params.min_depth= 10
• nextflow_config - Config default value correct: params.neg_control_pct_threshold= 10
• nextflow_config - Config default value correct: params.neg_ctrl_substrings= neg,ntc,blank,en
• nextflow_config - Config default value correct: params.nextclade_dataset_name= nextstrain/measles/N450/WHO-2012
• nextflow_config - Config default value correct: params.nextclade_dataset_tag= 2025-03-26--11-47-13Z
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.validate_params= true
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• readme - README Zenodo placeholder was replaced with DOI.
• pipeline_todos - No TODO strings found
• plugin_includes - No wrong validation plugin imports have been found
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (0 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: branch.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: ci_workflow.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
• local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
• base_config - conf/base.config found and not ignored.
• modules_config - conf/modules.config found and not ignored.
• modules_config - FASTQC found in conf/modules.config and Nextflow scripts.
• modules_config - NANOQ found in conf/modules.config and Nextflow scripts.
• modules_config - FASTP found in conf/modules.config and Nextflow scripts.
• modules_config - BWAMEM2_MEM found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_CONSENSUS_ILLUMINA found in conf/modules.config and Nextflow scripts.
• modules_config - NEXTCLADE_DATASETGET found in conf/modules.config and Nextflow scripts.
• modules_config - NEXTCLADE_RUN_N450 found in conf/modules.config and Nextflow scripts.
• modules_config - NEXTCLADE_RUN_CUSTOM found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_DEPTH found in conf/modules.config and Nextflow scripts.
• modules_config - BEDTOOLS_COVERAGE found in conf/modules.config and Nextflow scripts.
• modules_config - QUAST found in conf/modules.config and Nextflow scripts.
• modules_config - CALCULATE_BAM_VARIATION found in conf/modules.config and Nextflow scripts.
• modules_config - STAGE_FILE_IRIDANEXT found in conf/modules.config and Nextflow scripts.
• modules_config - GENERATE_REF_INTERMEDIATES found in conf/modules.config and Nextflow scripts.
• modules_config - GENERATE_AMPLICON_BED found in conf/modules.config and Nextflow scripts.
• modules_config - SPLIT_AMPLICON_REGION found in conf/modules.config and Nextflow scripts.
• modules_config - PREDICT_GENOTYPE found in conf/modules.config and Nextflow scripts.
• modules_config - BWAMEM2_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - IVAR_TRIM found in conf/modules.config and Nextflow scripts.
• modules_config - PICARD_MARKDUPLICATES found in conf/modules.config and Nextflow scripts.
• modules_config - FREEBAYES found in conf/modules.config and Nextflow scripts.
• modules_config - PROCESS_ILLUMINA_VCF found in conf/modules.config and Nextflow scripts.
• modules_config - ARTIC_GET_MODELS found in conf/modules.config and Nextflow scripts.
• modules_config - MINIMAP2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - ALIGN_TRIM found in conf/modules.config and Nextflow scripts.
• modules_config - CLAIR3 found in conf/modules.config and Nextflow scripts.
• modules_config - ARTIC_VCF_MERGE found in conf/modules.config and Nextflow scripts.
• modules_config - PROCESS_NANOPORE_VCF found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_NORM found in conf/modules.config and Nextflow scripts.
• modules_config - VCF_TO_TSV found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_CONSENSUS found in conf/modules.config and Nextflow scripts.
• modules_config - ADJUST found in conf/modules.config and Nextflow scripts.
• modules_config - COMPARE_INTERNAL_DSID found in conf/modules.config and Nextflow scripts.
• modules_config - MAKE_SAMPLE_QC_CSV found in conf/modules.config and Nextflow scripts.
• modules_config - MAKE_FINAL_QC_CSV found in conf/modules.config and Nextflow scripts.
• modules_config - SAMPLE_AMPLICON found in conf/modules.config and Nextflow scripts.
• modules_config - AMPLICON_ found in conf/modules.config and Nextflow scripts.
• modules_config - AMPLICON_MULTIQC found in conf/modules.config and Nextflow scripts.
• modules_config - POSITIONAL_N_DEPTH found in conf/modules.config and Nextflow scripts.
• modules_config - PYSAMSTATS found in conf/modules.config and Nextflow scripts.
• modules_config - NORMALIZE_DEPTH_MATRIX found in conf/modules.config and Nextflow scripts.
• modules_config - MAKE_CUSTOM_REPORT found in conf/modules.config and Nextflow scripts.
• modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
• nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
• nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.2.1

Run details

• nf-core/tools version 3.2.1
• Run at 2026-07-02 14:53:59

[AhmedAbdalla4]

Thanks for the update!