Skip to content

tr: do not expand [c*n] repeats character by character - #14524

Open
TanbirRamim wants to merge 4 commits into
uutils:mainfrom
TanbirRamim:tr-repeat-count
Open

tr: do not expand [c*n] repeats character by character#14524
TanbirRamim wants to merge 4 commits into
uutils:mainfrom
TanbirRamim:tr-repeat-count

Conversation

@TanbirRamim

Copy link
Copy Markdown

A [c*n] repeat was expanded into n bytes up front, so tr '[a*9223372036854775808]' b died with capacity overflow and tr '[a*99999999999999]' b with an allocation failure before reading any input. A large count in SET2 did not allocate, but spun in a count() over the expansion.

The sets are now resolved as runs of a repeated character. The lengths that the existing checks need come from the runs, and the two sets are lined up run by run: a run in SET1 maps its character to whatever it lines up with in SET2, and only the last mapping matters, so one pair per run boundary is enough. The vectors handed to the translate, delete and squeeze operations keep the same mappings in the same order and the same set membership, just without the repetition. I compared the old and new binaries on a few thousand combinations of sets and flags and got identical output for all of them.

Fixes #14420

A `[c*n]` repeat was expanded into `n` bytes up front, so a large count
aborted with a capacity overflow or an allocation failure before any
input was read, and a large count in SET2 spun in a `count()` over the
expansion.

Resolve the sets as runs of a repeated character instead. The lengths the
existing checks need come from the runs, and the two sets are lined up
run by run: a run in SET1 maps its character to whatever it lines up with
in SET2, and only the last mapping matters, so one pair per run boundary
is enough. The vectors handed to the translate, delete and squeeze
operations keep the same mappings in the same order and the same set
membership, just without the repetition.

Fixes uutils#14420
Copilot AI lite review requested due to automatic review settings September 12, 2026 20:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The implementation has a length-alignment issue, and one regression test is invalid on supported 32-bit targets.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates tr to handle large [c*n] repeats as compact runs instead of eagerly expanding them, with regression coverage.

Changes:

  • Implements run-based set resolution and mapping.
  • Preserves translation, deletion, and squeeze behavior.
  • Adds large-repeat regression tests.
File summaries
File Summary
src/uu/tr/src/operation.rs Implements run-based processing. Moderate issue: saturated lengths can misalign positions beyond usize::MAX.
tests/by-util/test_tr.rs Adds large-repeat tests. Moderate issue: a repeat-count test must be gated to 64-bit targets.
Review details

Suppressed comments (2)

src/uu/tr/src/operation.rs:362

  • Using usize::MAX as the prefix length does not mean “the whole set” once cumulative repeat lengths reach that value: remaining becomes zero and later runs are skipped. For example, tr -c '[a*18446744073709551614]bc' x incorrectly translates c, even though c is in SET1. Compute the complement over all runs separately from finite-prefix truncation.
            Self::complement_of_prefix(&set1, usize::MAX)

src/uu/tr/src/operation.rs:377

  • set1_len and set2_len have already been saturated before this subtraction, so [c*] padding is wrong when fixed lengths cross usize::MAX. With SET1 [a*18446744073709551615]b and SET2 [x*18446744073709551614][y*]z, the star should repeat once, but this computes zero and maps a to z instead of y; retain exact or wider lengths for the compensation calculation.
        let star_compensate_len = set1_len.saturating_sub(set2_len);
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/uu/tr/src/operation.rs Outdated
Comment thread tests/by-util/test_tr.rs
Review pointed out that summing the run lengths with saturating
arithmetic made positions past usize::MAX collide: the complement could
skip characters at the end of SET1, a [c*] star could pad too little, and
misaligned classes could pass the alignment check. Sum the lengths as
u128 instead, and stop using usize::MAX as a "whole set" sentinel.

The tests with repeat counts that only parse on 64-bit targets are now
gated to those targets.
Copilot AI review requested due to automatic review settings September 12, 2026 21:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Preserve widened run lengths through pairing to avoid incorrect SET1/SET2 mappings.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/uu/tr/src/operation.rs:374

  • This conversion silently clamps a valid [c*] padding length when the SET1 expansion exceeds usize::MAX by more than usize::MAX. That shifts every following SET2 run (including character classes) earlier and can reject or apply an incorrect mapping; keep the run length widened (for example, as u128) through pairing instead of saturating it here.
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/misc/tty-eof (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/timeout/timeout-group (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/basenc/bounded-memory is now being skipped but was previously passing.

The test binary is 64-bit but it drives a wasm32 coreutils, where these
counts do not fit in usize and are rejected as invalid, so the pointer
width guard alone does not exclude them.
Copilot AI review requested due to automatic review settings September 12, 2026 23:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 12, 2026 23:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tr: a large [c*n] repeat count is materialized unbounded — capacity overflow panic / allocation abort (exit 134)

2 participants