Fix nonzero bool nonstandard bytes - #3055
Conversation
`NonZeroIndicator` compared against `inputT(0)` and the `Cumsum1D` factories used `NoOpTransformer`; for `bool` both fold into a raw byte load, so the scan summed byte values instead of 0/1. A mask stored as [0, 1, 2, 255, 0, 1] reported 259 non-zeros instead of 4, affecting `nonzero`, `where`, `extract`, `place` and `repeat`. Cast bool via `sycl::bit_cast<std::uint8_t>` / `CastTransformer`, as `convert_impl` already does (IntelPythongh-2121).
C++ may fold a bool comparison into a raw byte load, so a byte other than 0x00/0x01 compared unequal to a normalized True, ordered by its byte value, and leaked into computed bool results. Such a byte arises when a buffer is written through a raw pointer or viewed from integer data. Add `normalize_bool` and apply it where a bool is read from memory: elementwise operand loads (bool is excluded from the vector paths, which cannot normalize per element), the `convert_impl` same-type branch, the search-reduction loads, the `isin` equality test and the argsort projection. Add bool comparators for the merge-sort path. `sort` now orders False before True rather than reproducing NumPy's raw byte order, which would carry garbage bytes through a sort.
|
Can one of the admins verify this patch? |
ndgrigorian
left a comment
ndgrigorian
left a comment
There was a problem hiding this comment.
I've looked over the changes, seems like a welcome fix for a previously unnoticed bug, LGTM
|
@abagusetty seems that the tests fail with the open-source compiler, interestingly enough. Possible bug in the nightly DPC++? |
|
coverage: 78.465% (-0.02%) from 78.489% — abagusetty:fix-nonzero-bool-nonstandard-bytes into IntelPython:master |
`bool` takes a single radix pass, so the bucket index is `byte & 0xF`. An unnormalized byte whose low nibble is zero (0x10, 0x80, 0xF0) bucketed as False and sorted before True elements. Normalize in `order_preserving_cast`, where every radix path converges, and take the argument by reference so a copy cannot let the compiler assume a 0/1 byte.
No bug in nightly. Interestingly, it is doing great. The issue was UB in the PR that the two compilers are treating it differently. A bool whose byte isnt |
4 participants
Boolean arrays may contain non-zero bytes like
0x02or0xFFthat NumPy treats as True but dpnp does not.Fixes #3054