Repository navigation
Conversation
Add GitHub Actions CI for automated GPU snapshot updates and update C…
…arity and consistency
…g bwameth container, and raising Nextflow version requirement. Add GitHub Actions CI for GPU tests snapshot update.
…rimer on Bisulfite Sequencing and Three-Base Aligners. Standardize references to the new documentation structure.
modules update + minor bug fix + doc link fix
Add wave to ARM Megatests
bump version 4.1.0
…xtflow_schema.json and nextflow.config, and enhance usage documentation with examples for BWA-MEM2 algorithm.
…dexing and alignment, including `--use_mem2` parameter and updates to related workflows and modules.
Adding test for mem2 mode
…, including path adjustments and minor documentation fixes
… targeted sequencing options
Brings in dev's rastair strict-parser fix (#618, clears the pre-commit check) and Trim Galore 2.3.0 (#617). The 8 pipeline snapshots are resolved to the template-branch versions here as placeholders; they are regenerated in the following commit against the merged modules (multiqc 1.34 + Trim Galore 2.3.0), since neither branch's snapshots match the merged combination.
…im Galore 2.3.0) After merging dev (Trim Galore 2.3.0, #617) into the template branch (multiqc 1.34 etc.), all CPU pipeline snapshots were regenerated against the merged modules on Nextflow 25.04.0 + docker; every file passes. GPU (bwameth_use_gpu) regenerated separately via the update-gpu-snapshot bot.
The container_configs lint check flagged conf/containers_*.config as out of date after the dev merge (added the missing Bismark 3.1.0 process container mappings). Regenerated via 'nf-core pipelines lint --fix container_configs' (nf-core 4.0.2, Nextflow 25.04.6); nf-core lint now passes with 0 failures.
The combined_index.nf.test snapshot was frozen with Trim Galore 0.6.10 before #617 (TG 2.3.0) landed on dev. Regenerated on Nextflow 25.04.0 + docker; all 5 scenarios pass (directional, non-directional, PBAT, HISAT2, loud-fail).
Important! Template update for nf-core/tools v4.0.2
…merge) #606 (template v4.0.2, multiqc 1.32->1.34) landed on dev and was merged into this branch, re-staling the combined_index snapshot. Regenerated on Nextflow 25.04.0 + docker against the merged modules (multiqc 1.34 + Trim Galore 2.3.0); all 5 scenarios pass.
Add opt-in Bismark combined-index alignment (`--combined_index`)
Resolve template sync v4.0.3 conflicts: - README.md: keep the >=25.04.0 Nextflow badge (pipeline is not 25.10 strict-clean), take the 4.0.3 template badge. - ro-crate-metadata.json: keep dev's richer author/maintainer metadata (the bot-regenerated crate lost it); crate is regenerated at release anyway.
Important! Template update for nf-core/tools v4.0.3
Repins all seven modules/nf-core/bismark/* from 7d264419 (the bismark 3.1.0 bump) to c565f1b2, and retires the local bismark-align.diff patch. The patch is not being cleaned up, it is being removed because it blocks the change: its combined-index branch demoted the classic path's `--multicore` block to an `else if`, so importing nf-core/modules#12385 without removing the patch would ship the new -p model and disable it on every non-combined run. Upstream's general model subsumes the patch exactly. For every argument set conf/modules/bismark_align.config can emit, `--combined_index` resolves to n_instances = 1, so pthreads == task.cpus -- the same value the patch computed, with the same `>= 2` floor. The combined-index rows therefore do not change. What does change, at the CI cap of 4 cpus: directional Bowtie 2 and HISAT2 go from no threading flag to `-p 2`. At the production process_high of 12 cpus, directional goes from `--multicore 4` to `-p 6`, non-directional from `--multicore 2` to `-p 3`, and bismark_hisat from an effective `-p 4` (bismark remaps --multicore for HISAT2) to `-p 6`. Bowtie 2 output is byte-identical under `-p N --reorder`; HISAT2 is thread-count dependent, so bismark_hisat results can shift for real users -- hence the CHANGELOG note. No snapshot is expected to move. Upstream's own module tests snapshot a reads MD5, made this exact no-flag -> `-p 2` transition at cpus = 4, and their tests/main.nf.test.snap blob is identical either side of #12385 (validated by green CI across docker, singularity and conda). Also adds a published-report assertion to the bismark and bismark_hisat pipeline tests so the emitted -p value is checked by CI rather than by hand. The size guard in front of it matters: Groovy's every{} is true on an empty list, so without it the assertion would pass when no report was found. Drops a now-misleading clause from the unreleased combined-index changelog entry, which described `-p` as specific to combined-index mode. Imported with `git checkout <sha> -- modules/nf-core/bismark/` rather than `nf-core modules update`, which cascades into unrelated modules. The prettier pre-commit hook was bypassed: upstream dev's modules.json is already prettier-dirty under the pinned version, so running the hook would have reformatted ~117 lines unrelated to this change.
Two conda shards fail on this branch because conda builds a different BS_CT.1.ht2 / BS_GA.1.ht2 / BS_combined.1.ht2 than the committed snapshots expect. Nothing in the pipeline changed: .2 through .8 of every index, all the .bt2 files, the converted FASTAs and every alignment and methylation output match byte-for-byte, on every engine. The mismatch is a bioconda hisat2 rebuild, not a regression here. It is deterministic (a re-run produced byte-identical hashes), the conda value has never been committed to this repo, and #616 ran 30 conda jobs green on 24 July with the same expected hashes. Since a single snapshot cannot satisfy conda and the containers at once, the .1.ht2 bytes are not something these tests can assert. Scope is deliberately narrow: only the .1.ht2 of each index is ignored, so 35 .2-.8 hashes and 66 .bt2 hashes stay asserted, and because stable_name does not consult .nftignore the five index files are still checked for existence and naming — only their bytes are dropped. The five snapshot entries were removed by hand rather than regenerated: no local run can produce CI-comparable hashes here (arm64, and the container registry is unreachable from this Docker VM), and a wrong edit fails loudly in CI. Verified the ignore glob against java.nio.file.PathMatcher in both directions before applying it. The remaining red check, GPU Test | singularity, is unrelated and already red on dev: apptainer cannot create a user namespace on that runner and the job dies at the first process of a bwameth test.
…12385 Sync bismark/* modules to pick up intra-instance -p threading
Add `manifest.diagram` to `nextflow.config`
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Records the merge of
devinto the template branch. Reviewed by PR 2.