Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,25 @@ Sections commonly used: Features, Bug fixes, Other changes.

### Bug fixes

- **`CellReads.stats` as STARsolo writes it.** `--soloCellReadStats` now
takes STAR's value, `Standard`; `CB`, which rustar-aligner took before,
is kept as a deprecated alias. `--genomeChrSetMitochondrial` defaults to
STAR's `chrM M MT` instead of naming nothing. The file's values now follow
STAR's (`SoloReadFeature_record.cpp`, `SoloReadFeature_inputRecords.cpp`):
each feature keeps its own table, so `Gene` and `GeneFull` no longer print
the same rows, and the region columns come from the feature's own overlap
type (`exonic` for a `Gene` read with a gene, nothing for `GeneFull`)
instead of CellRanger's positional regions. Every read is counted under
`cbMatch`, including those whose barcode or UMI was rejected; one-mismatch
corrections and 1MM_multi barcodes are resolved with the run's exact counts,
as the matrix resolves them, so `cbMMunique`/`cbMMmultiple` and the rows
match; `featureM` and `countedM` are filled; and `nUMImulti`/`nGenesMulti`
come from the first `--soloMultiMappers` matrix. Rows are folded in input
order by the in-order consumer, so their order no longer depends on thread
timing; it is still not STAR's order (DIVERGENCE.md 3.2). On CellRanger's tiny 10x data every value matches STARsolo 2.7.11b
except on the 3 barcodes of 3 reads rustar-aligner maps and STAR leaves
unmapped.

- **STARsolo's stats files as STAR writes them.** `Solo.out/Barcodes.stats`,
`<feature>/Features.stats` and `<feature>/UMIperCellSorted.txt` were not
written, SJ and Velocyto had no `Summary.csv`, and the Gene/GeneFull
Expand Down
12 changes: 5 additions & 7 deletions DIVERGENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,17 +225,15 @@ There are none at present. The four yeast PE reads once listed here now match ST

### 3.2 `CellReads.stats` row order

**What STAR does.** `--soloCellReadStats CB` emits its rows by iterating a libc++ `std::unordered_map`, so the order is a hash-table walk rather than a sort. At the map sizes this produces, libc++ chains new entries at the head of their bucket and walks buckets in order, which comes out as the reverse of each barcode's first appearance in read order.
**What STAR does.** `--soloCellReadStats Standard` writes its rows by iterating `readFlagCounts.flagCounts`, a `std::unordered_map` keyed by whitelist index and reserved at `nCB*3/2` buckets (`SoloCommon.h`, `SoloFeature_countCBgeneUMI.cpp`, `SoloFeature_statsOutput.cpp`), so the order is a hash-table walk rather than a sort.

**What rustar-aligner does.** Emits that same reverse-first-appearance order, including across threads: the per-read accumulator merges in read order, so a threaded run writes the same file as a serial one.
**What rustar-aligner does.** Writes the rows in reverse order of each barcode's first appearance in the reads. Each read's entry is folded by the in-order consumer of the alignment loop, so a threaded run writes the same file as a serial one. A read whose barcode is resolved only at the end (a one-mismatch correction under `oneExact`, or a 1MM_multi barcode) counts as appearing where it was read.

**Why.** Reproducing the order where it is reproducible costs nothing and keeps a byte-comparison against STAR usable on the sizes where it can work at all.
**Measured.** On CellRanger's tiny 10x data (6,982 barcodes), STAR 2.7.11b (macOS build) writes the same order at 1 and at 8 threads, and rustar-aligner's order matches it only on the first rows. A model of a libc++ walk (identity hash, the bucket a key modulo a prime, a node for an empty bucket at the front of the list), fed rustar-aligner's first-appearance order, does not reproduce STAR's order for any bucket count up to 12,000 or any of three barcode keys (sorted-whitelist rank, file position, packed sequence). What sets STAR's order is not settled, and it may differ between standard libraries.

**Impact.** Past libc++'s load factor the map rehashes, and the order then depends on the bucket count, which depends on how many distinct barcodes were seen; beyond that size the order diverges. The **values never do** — only which line they appear on. Reading the file by barcode rather than by position is unaffected either way.
**Why.** The values are what the file reports; the row order is an artefact of STAR's container.

**Source.** `src/solo/cell_reads.rs`, locked by `rows_are_emitted_in_reverse_first_appearance_order`. STAR: `SoloFeature_statsOutput.cpp`.

---
**Impact.** Every value matches (on tiny 10x, all but the 3 barcodes of 3 reads rustar-aligner maps and STAR leaves unmapped); only the line each barcode is on differs. Read the file by barcode, not by position. A byte comparison against STAR's file does not work; sort both by barcode first.

### 3.3 SAM/BAM header program lines

Expand Down
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/cli-parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Run `rustar-aligner --help` for the full machine-generated listing.
| `--genomeSAindexNbases` | `14` | Length of the SA pre-indexing string (log2). Lower for small genomes. |
| `--genomeChrBinNbits` | `18` | Log2 of chromosome bin size. |
| `--genomeSAsparseD` | `1` | SA sparsity (higher = less RAM, slower mapping). |
| `--genomeChrSetMitochondrial` | `chrM M MT` | Chromosomes counted as mitochondrial, for the `mito` column of `CellReads.stats`. `-` names none. |

## Read input

Expand Down Expand Up @@ -162,6 +163,7 @@ Run `rustar-aligner --help` for the full machine-generated listing.
| `--quantGeneCoverage` | `No` | rustar-aligner extension. `Yes` writes per-gene exonic coverage to `GeneCoverage.out.bin` for DegNorm. See [DegNorm](/rustar-aligner/guides/degnorm/). |
| `--degNormSampleId` | basename of `--outFileNamePrefix` | rustar-aligner extension. Sample name stored in `GeneCoverage.out.bin`. |
| `--degNormCoverageFiles` | — | rustar-aligner extension. Two or more `GeneCoverage.out.bin` files: runs the DegNorm merge (no reads, no `--runMode`) and writes `DegNorm.out/`. Tuning: `--degNormIter`, `--degNormNmfIter`, `--degNormDownsampleRate`, `--degNormMinimaxCoverage`, `--degNormSkipBaselineSelection`, `--degNormBins`, `--degNormMinHighCoverage` (see the guide). |
| `--soloCellReadStats` | `None` | `Standard` writes `Solo.out/<feature>/CellReads.stats` as STARsolo does: per cell barcode, how its reads' barcodes matched (`cbMatch`, `cbPerfect`, `cbMMunique`, `cbMMmultiple`), whether they mapped uniquely (`genomeU`/`genomeM`), reached one or several genes of the feature (`featureU`/`featureM`), their region (`exonic` for `Gene`; `GeneFull` sets none), `mito`, whether they reached the unique or multi-gene matrix (`countedU`/`countedM`), and the cell's UMI and gene totals. Reads whose barcode or UMI did not pass are summed in the `CBnotInPasslist` row. Rows come in reverse first-appearance order, which is not STAR's order (see the divergence notes); the values match, so read it by barcode. `CB` is a deprecated alias of `Standard`; STAR rejects it. |
| `--soloCellRangerSummary` | `No` | rustar-aligner extension. `Yes` writes `Solo.out/<feature>/CellRanger.summary.csv` for `Gene` and `GeneFull` (both needed): the fractions of all reads that map uniquely in exonic, intronic or intergenic regions, or antisense to a gene, under CellRanger's metric names. STARsolo's `Summary.csv` is unchanged. |

## Two-pass mode
Expand Down
11 changes: 11 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2522,6 +2522,7 @@ fn align_reads_solo<W: AlignmentWriter + ?Sized>(
per_feature: Vec<crate::solo::FeatureOutcome>,
sj: crate::solo::SjRecords,
velocyto: Option<crate::solo::VelocytoRecord>,
cell_read: Option<crate::solo::cell_reads::CellReadEntry>,
}

info!("STARsolo: aligning cDNA reads and quantifying barcodes...");
Expand Down Expand Up @@ -2575,6 +2576,7 @@ fn align_reads_solo<W: AlignmentWriter + ?Sized>(
(0..n_feat).map(|_| Vec::new()).collect();
let mut no_feature_multi: Vec<Vec<SoloMultiRecord>> =
(0..n_feat).map(|_| Vec::new()).collect();
let mut cell_reads: Vec<crate::solo::cell_reads::CellReadEntry> = Vec::new();
for result in products {
let product = result?;
writer.write_batch(&product.sam_records.records)?;
Expand All @@ -2597,7 +2599,9 @@ fn align_reads_solo<W: AlignmentWriter + ?Sized>(
}
sj_batch.append(product.sj);
velo_batch.extend(product.velocyto);
cell_reads.extend(product.cell_read);
}
solo.fold_cell_reads(cell_reads);
for (fi, recorder) in solo.recorders.iter().enumerate() {
recorder.extend(
std::mem::take(&mut feat_records[fi]),
Expand Down Expand Up @@ -2710,6 +2714,7 @@ fn align_reads_solo<W: AlignmentWriter + ?Sized>(
per_feature: outcome.per_feature,
sj: outcome.sj,
velocyto: outcome.velocyto,
cell_read: outcome.cell_read,
});
}

Expand Down Expand Up @@ -2873,6 +2878,7 @@ fn align_reads_solo<W: AlignmentWriter + ?Sized>(
per_feature: outcome.per_feature,
sj: outcome.sj,
velocyto: outcome.velocyto,
cell_read: outcome.cell_read,
})
})
.collect();
Expand Down Expand Up @@ -2971,6 +2977,7 @@ fn align_reads_solo_pe<W: AlignmentWriter + ?Sized>(
per_feature: Vec<crate::solo::FeatureOutcome>,
sj: crate::solo::SjRecords,
velocyto: Option<crate::solo::VelocytoRecord>,
cell_read: Option<crate::solo::cell_reads::CellReadEntry>,
}

info!("STARsolo: aligning 5' paired-end reads and quantifying barcodes...");
Expand Down Expand Up @@ -3012,6 +3019,7 @@ fn align_reads_solo_pe<W: AlignmentWriter + ?Sized>(
(0..n_feat).map(|_| Vec::new()).collect();
let mut no_feature_multi: Vec<Vec<SoloMultiRecord>> =
(0..n_feat).map(|_| Vec::new()).collect();
let mut cell_reads: Vec<crate::solo::cell_reads::CellReadEntry> = Vec::new();
for result in products {
let product = result?;
writer.write_batch(&product.sam_records.records)?;
Expand All @@ -3034,7 +3042,9 @@ fn align_reads_solo_pe<W: AlignmentWriter + ?Sized>(
}
sj_batch.append(product.sj);
velo_batch.extend(product.velocyto);
cell_reads.extend(product.cell_read);
}
solo.fold_cell_reads(cell_reads);
for (fi, recorder) in solo.recorders.iter().enumerate() {
recorder.extend(
std::mem::take(&mut feat_records[fi]),
Expand Down Expand Up @@ -3398,6 +3408,7 @@ fn align_reads_solo_pe<W: AlignmentWriter + ?Sized>(
per_feature: outcome.per_feature,
sj: outcome.sj,
velocyto: outcome.velocyto,
cell_read: outcome.cell_read,
})
})
.collect();
Expand Down
32 changes: 24 additions & 8 deletions src/params/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1414,15 +1414,20 @@ pub struct Parameters {
#[arg(long = "soloCBmatchWLtype", default_value = "1MM_multi")]
pub solo_cb_match_wl_type: String,

/// `CB`: write `Solo.out/<feature>/CellReads.stats`, a per-cell-barcode
/// summary of what happened to the reads carrying it. `None` (the default)
/// writes nothing.
/// `Standard`: write `Solo.out/<feature>/CellReads.stats`, a
/// per-cell-barcode summary of what happened to the reads carrying it.
/// `None` (the default) writes nothing. `CB` is a deprecated alias of
/// `Standard` (STAR rejects it).
#[arg(long = "soloCellReadStats", default_value = "None")]
pub solo_cell_read_stats: String,

/// Chromosome names treated as mitochondrial, for the `mito` column of
/// `CellReads.stats`. `-` (the default) names none.
#[arg(long = "genomeChrSetMitochondrial", num_args = 1.., default_values_t = vec!["-".to_string()])]
/// `CellReads.stats`. STAR's default is `chrM M MT`; `-` names none.
#[arg(
long = "genomeChrSetMitochondrial",
num_args = 1..,
default_values_t = ["chrM", "M", "MT"].map(String::from)
)]
pub genome_chr_set_mitochondrial: Vec<String>,
/// Two-column `CB cluster` file assigning cells to clusters, for
/// `--soloFeatures Transcript3p`.
Expand Down Expand Up @@ -1537,6 +1542,12 @@ impl Parameters {
}
}

/// `--soloCellReadStats Standard` (or its deprecated alias `CB`): write
/// `CellReads.stats`.
pub fn solo_cell_read_stats_yes(&self) -> bool {
matches!(self.solo_cell_read_stats.as_str(), "Standard" | "CB")
}

/// Whether the run has to track STAR's readInfo, the per-read (cell,
/// corrected UMI) pair that UMI collapsing fills and the sorted-BAM writer
/// turns into `CB`/`UB`. `CB_samTagOut` corrects the barcode inline instead,
Expand Down Expand Up @@ -2601,12 +2612,17 @@ impl Parameters {
SOLUTION: rerun with --soloUMIfiltering MultiGeneUMI_CR --soloUMIdedup 1MM_CR",
));
}
// --soloCellReadStats: `CB` is the only value STAR defines.
if !matches!(params.solo_cell_read_stats.as_str(), "CB" | "None") {
// --soloCellReadStats: STAR defines `Standard` and `None`
// (`ParametersSolo.cpp:441-451`); `CB`, which rustar-aligner took
// before, stays as a deprecated alias of `Standard`.
if !matches!(
params.solo_cell_read_stats.as_str(),
"Standard" | "None" | "CB"
) {
return Err(command.error(
ErrorKind::InvalidValue,
format!(
"unknown --soloCellReadStats '{}'; expected CB or None",
"unknown --soloCellReadStats '{}'; expected Standard or None",
params.solo_cell_read_stats
),
));
Expand Down
Loading
Loading