From 32995de1c1fbd7cc3d79c9a6f974fe841256016f Mon Sep 17 00:00:00 2001 From: LinxISA Automation Date: Thu, 29 Oct 2026 15:14:57 +0800 Subject: [PATCH] Add typed PredicateCell logical composition and selection Adopt PTO spec #406 for logical PredicateCell results, mixed Numeric U8 outputs, basis-independent selection, CELL-aligned typed views and closed two-writer concatenation. Keep indexed memory restricted to M32 and document matching LLVM requirements. --- docs/tileop-usage/element-if.md | 7 +- docs/tileop-usage/options.md | 5 +- docs/tileop-usage/predicate-adapters.md | 61 ++++- include/common/pto_predicate.hpp | 327 +++++++++++++++++++++++- test/test_predicate_adapters.py | 132 ++++++++++ 5 files changed, 513 insertions(+), 19 deletions(-) diff --git a/docs/tileop-usage/element-if.md b/docs/tileop-usage/element-if.md index 52afc62..4bf71f5 100644 --- a/docs/tileop-usage/element-if.md +++ b/docs/tileop-usage/element-if.md @@ -104,9 +104,10 @@ predicate::atomic_add(destination, byte_offsets, values, active); 把这些调用放进循环就能编译。 两条路径使用相同架构中的 predicate/ExecutionMask 概念，但公开支持边界 -不同。显式 adapters 首版只覆盖 Local VEC CUBE_M32、静态有效区域和 -FP32/S32/U32，尚无 predicate 逻辑组合；本页 element-if 已支持的组合条件 -不能据此推广到 `Predicate`。显式 `atomic_add` 使用 MSCATTER_ADD +不同。显式 adapters 支持 Local VEC CUBE_M16/M32 的静态比较、谓词逻辑 +组合、选择与受限 typed views/拼接；indexed memory 仍仅覆盖 M32， +FP32/S32/U32。详细类型与形状边界见独立 adapters 文档，不能将本页 +pragma 支持范围与 `Predicate` 混为一谈。显式 `atomic_add` 使用 MSCATTER_ADD ExecutionMask，本页 histogram 示例的已验收 profile 则使用 MGATHER.ADD， 不能互相替代其 dtype、memory-order、返回值或验证结论。 diff --git a/docs/tileop-usage/options.md b/docs/tileop-usage/options.md index b977ec1..0d0ddb8 100644 --- a/docs/tileop-usage/options.md +++ b/docs/tileop-usage/options.md @@ -18,7 +18,10 @@ `predicate::compare_scalar` 的强类型结果可以直接供给 `predicate::scatter`，或作为 `predicate::atomic_add` 的 ExecutionMask。`Invert` 是编译期布尔参数，默认 false，不属于 -`fixp::Options`。当前支持边界、byte-offset 单位和未支持的逻辑组合见 +`fixp::Options`。显式 `logical_and/or/xor` 组合产生 U8-basis PredicateCell， +`select/select_scalar` 可选择不同 basis 的 Numeric payload；`subview` 保留 +basis，`concatenate` 只返回已封闭 parent。混合逻辑 `logical_numeric<6/7/8>` +返回 Numeric U8 而非谓词。当前支持边界和 CELL/byte-offset 单位见 [PredicateCell adapters](predicate-adapters.md)。 ## 1. 快速开始 diff --git a/docs/tileop-usage/predicate-adapters.md b/docs/tileop-usage/predicate-adapters.md index d3aac40..a422d0e 100644 --- a/docs/tileop-usage/predicate-adapters.md +++ b/docs/tileop-usage/predicate-adapters.md @@ -17,6 +17,10 @@ sufficient. Keep the new TileOP headers and matching compiler together. Do not replace headers in an older installed SDK before that compiler dependency is available. +The spec #406 composition/view APIs additionally require matching LLVM +PredicateCell result-identity propagation. The earlier adapter compiler alone +does not establish that contract, even if a simple straight-line case compiles. + Include `common/pto_tileop.hpp` when targeting Linx. The experimental `pto::predicate` API keeps a comparison result distinct from ordinary numeric Tiles and packed GPR predicates: @@ -67,17 +71,19 @@ tree that takes precedence over the selected headers. ## Carrier and consumer contract -The first implementation admits static valid shapes, Local VEC CUBE_M32, +The adapters admit static valid shapes, Local VEC CUBE_M16/M32, FP32/S32/U32 comparison and value operands, and S64/U64 **byte displacements**. -The value type, valid region and layout must match the comparison basis. +Valid region and layout must match the predicate, but its comparison basis +does not constrain the FP32/S32/U32 payload type. Indexed memory remains +CUBE_M32-only. M16 supports comparison, logical composition, selection, +typed subviews and concatenation, not indexed memory. Offsets may have a different physical capacity, but must cover that valid region. `destination` must have the same element type as `values`. `scatter` binds the PredicateCell directly to the explicit MSCATTER_MASK operand. It does not convert it with TCVT or reinterpret it as an ordinary numeric U8 Tile. This follows the executable -`IndexedTLSUPredicateDescriptorLegal` helper; the instruction-page wording -discrepancy is tracked in PTO-ISA/pto-spec #349. +`IndexedTLSUPredicateDescriptorLegal` helper and the contracts in spec #406. `atomic_add` requires the compiler builtin `ew_mscatter_add_predicate` and emits MSCATTER_ADD with a PredicateCell ExecutionMask. `true` inverts the @@ -87,11 +93,52 @@ addresses, probe memory or enqueue atomic RMWs. Structural legality checks still apply. Replacing inactive addends with zero is not equivalent. `Predicate` cannot be default-constructed by callers or imported from -an arbitrary U8 Tile. Its carrier is intentionally private. Ordinary TAND is -not a PredicateCell logical operator. Predicate composition, GPR selection, -first-set-bit operations, dynamic valid shapes, M16 and cross-call predicate +an arbitrary U8 Tile. Its carrier is intentionally private. GPR selection, +first-set-bit operations, dynamic valid shapes and cross-call predicate ABI/spill support are not provided by this experimental interface. Keep producer and consumers together in an optimized, inlined region. +## Logical composition and selection + +```cpp +auto lower = predicate::compare_scalar(values, low); +auto upper = predicate::compare_scalar(values, high); +auto active = predicate::logical_and(lower, upper); +predicate::select(result, active, true_values, false_values); +predicate::select_scalar(result, active, true_values, 0u); +predicate::atomic_add(destination, byte_offsets, values, active); +``` + +`logical_and`, `logical_or` and `logical_xor` use U8 TAND/TOR/TXOR. +Both inputs must have identical valid and physical shapes and layouts, but +their original comparison bases may differ. The result is a private +PredicateCell with **U8 basis**, not a Numeric U8 Tile. Numeric inputs are not +implicitly admitted. `logical_numeric<6/7/8>(predicate, numeric)` explicitly +combines a predicate with same-shape S8/U8 Numeric data and returns a **Numeric +U8 Tile**, which cannot be passed as a typed predicate consumer argument. + +`select` selects between Numeric true/false Tiles; `select_scalar` selects +between a Numeric true Tile and a raw scalar false value. The predicate basis +may differ from the payload type. Valid shapes/layouts must still match. + +## Typed views and closed assembly + +`subview(predicate)` materializes a typed TMOV with +B.SUBVIEW. Offsets are in **128-byte CELL units**, not elements or bytes. +Columns must cover complete U8 CELLs (4 columns for M32, 8 for M16), fit the +source valid region, and use at most 64KB view capacity. Basis is preserved. + +`concatenate(left, right)` emits INIT/LAST B.ASSEMBLE writers and returns only +the closed parent. Both inputs must be U8-basis logical results with identical +full physical rows, complete equal-size valid column regions and layout; +writer capacity is at most 64KB. The output joins their columns. For example, +an M32 32x8 logical result can be split into `subview<0, 4>` and +`subview<1, 4>`, then concatenated. Arbitrary OPEN-parent mutation is not exposed. + +These calls do not request an ExecutionMask on the logical writer or expose +MERGE. ZERO/MERGE, canonical-byte definedness and Q-generation pinning across +wait/trap recovery remain model/runtime requirements; object tests do not +prove those semantics or a complete TopK kernel. + Object/assembly compilation checks are not model execution or TopK numerical validation. No ISA or model legality checks are relaxed by this interface. diff --git a/include/common/pto_predicate.hpp b/include/common/pto_predicate.hpp index 0d450d2..4d1767c 100644 --- a/include/common/pto_predicate.hpp +++ b/include/common/pto_predicate.hpp @@ -7,17 +7,55 @@ namespace pto { namespace predicate { template class Predicate; + +template +using LogicalBasis = Tile; + +template +__attribute__((always_inline)) inline auto +logical(const Predicate &, const Predicate &) + -> Predicate>; + +template +__attribute__((always_inline)) inline auto subview(const Predicate &) + -> Predicate>; + +template +__attribute__((always_inline)) inline auto +concatenate(const Predicate &, const Predicate &) + -> Predicate>; + +template +__attribute__((always_inline)) inline auto +logical_numeric(const Predicate &, const Numeric &) + -> LogicalBasis; + +template +__attribute__((always_inline)) inline void +select(Value &, const Predicate &, const Value &, const Value &); + +template +__attribute__((always_inline)) inline void +select_scalar(Value &, const Predicate &, const Value &, + typename Value::DType); + template __attribute__((always_inline)) inline Predicate compare_scalar(Basis &source, typename Basis::DType scalar); template class Predicate { static_assert(Basis::Loc == Location::Vec && - Basis::BFractal == BLayout::CubeM32, - "Predicate currently requires a Local VEC CUBE_M32 basis"); + (Basis::BFractal == BLayout::CubeM32 || + Basis::BFractal == BLayout::CubeM16), + "Predicate requires a Local VEC CUBE_M16/M32 basis"); static_assert(Basis::ValidRow > 0 && Basis::ValidCol > 0, "Predicate currently requires static valid dimensions"); - static constexpr int StorageRows = 32; + static constexpr int StorageRows = + Basis::BFractal == BLayout::CubeM16 ? 16 : 32; static constexpr int MinimumColumns = 128 / StorageRows; static constexpr int StorageColumns = Basis::Cols < MinimumColumns ? MinimumColumns : Basis::Cols; @@ -29,6 +67,31 @@ template class Predicate { template friend __attribute__((always_inline)) inline Predicate compare_scalar(Source &, typename Source::DType); + template + friend __attribute__((always_inline)) inline auto + logical(const Predicate &, const Predicate &) + -> Predicate>; + template + friend __attribute__((always_inline)) inline auto + subview(const Predicate &) + -> Predicate>; + template + friend __attribute__((always_inline)) inline auto + concatenate(const Predicate &, const Predicate &) + -> Predicate>; + template + friend __attribute__((always_inline)) inline auto + logical_numeric(const Predicate &, const Numeric &) + -> LogicalBasis; + template + friend __attribute__((always_inline)) inline void + select(Value &, const Predicate &, const Value &, const Value &); + template + friend __attribute__((always_inline)) inline void + select_scalar(Value &, const Predicate &, const Value &, + typename Value::DType); template friend __attribute__((always_inline)) inline void scatter(Global &, const Index &, const Value &, const Predicate &); @@ -41,6 +104,10 @@ template class Predicate { using BasisTile = Basis; }; +#define PTO_PREDICATE_LAYOUT_ASM \ + ".if %c[Layout] == 31\nB.DATR CUBE_M16, DTYPE_NONE, Zero\n" \ + ".elseif %c[Layout] == 29\nB.DATR CUBE_M32, DTYPE_NONE, Zero\n.endif\n" + template __attribute__((always_inline)) inline Predicate compare_scalar(Basis &source, typename Basis::DType scalar) { @@ -55,12 +122,20 @@ compare_scalar(Basis &source, typename Basis::DType scalar) { Predicate result; asm volatile( "BSTART.TEPL 45, %D[Type]\n" + ".if %c[Layout] == 31\n" + ".if %c[Mode] == 0\nB.DATR CUBE_M16, DTYPE_NONE, Zero, EQ, RNONE\n" + ".elseif %c[Mode] == 1\nB.DATR CUBE_M16, DTYPE_NONE, Zero, NE, RNONE\n" + ".elseif %c[Mode] == 2\nB.DATR CUBE_M16, DTYPE_NONE, Zero, LT, RNONE\n" + ".elseif %c[Mode] == 3\nB.DATR CUBE_M16, DTYPE_NONE, Zero, GT, RNONE\n" + ".elseif %c[Mode] == 4\nB.DATR CUBE_M16, DTYPE_NONE, Zero, LE, RNONE\n" + ".else\nB.DATR CUBE_M16, DTYPE_NONE, Zero, GE, RNONE\n.endif\n" + ".elseif %c[Layout] == 29\n" ".if %c[Mode] == 0\nB.DATR CUBE_M32, DTYPE_NONE, Zero, EQ, RNONE\n" ".elseif %c[Mode] == 1\nB.DATR CUBE_M32, DTYPE_NONE, Zero, NE, RNONE\n" ".elseif %c[Mode] == 2\nB.DATR CUBE_M32, DTYPE_NONE, Zero, LT, RNONE\n" ".elseif %c[Mode] == 3\nB.DATR CUBE_M32, DTYPE_NONE, Zero, GT, RNONE\n" ".elseif %c[Mode] == 4\nB.DATR CUBE_M32, DTYPE_NONE, Zero, LE, RNONE\n" - ".else\nB.DATR CUBE_M32, DTYPE_NONE, Zero, GE, RNONE\n.endif\n" + ".else\nB.DATR CUBE_M32, DTYPE_NONE, Zero, GE, RNONE\n.endif\n.endif\n" "B.DIM zero, %c[Columns], ->LB0\n" "B.DIM zero, %c[Rows], ->LB1\n" "B.DIM zero, %c[PhysicalColumns], ->LB2\n" @@ -70,6 +145,7 @@ compare_scalar(Basis &source, typename Basis::DType scalar) { : [Source] "Tr"(source.data()), [Scalar] "r"(scalar), [Type] "i"(type_traits::TypeCode), [Mode] "i"(static_cast(Mode)), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), [Rows] "i"(Basis::ValidRow), [Columns] "i"(Basis::ValidCol), [PhysicalColumns] "i"(Basis::Cols), [Size] "i"(tile_type_traits::Storage::TileDType>::TilesizeCode)); @@ -78,6 +154,8 @@ compare_scalar(Basis &source, typename Basis::DType scalar) { template constexpr void validate_consumer() { + static_assert(Basis::BFractal == BLayout::CubeM32, + "Predicate indexed memory currently requires CUBE_M32"); static_assert(Index::Loc == Location::Vec && Value::Loc == Location::Vec, "Predicate consumers require Local VEC operands"); static_assert(Index::BFractal == Basis::BFractal && @@ -95,8 +173,6 @@ constexpr void validate_consumer() { std::is_same_v || std::is_same_v, "Predicate consumers currently support FP32/S32/U32 values"); - static_assert(std::is_same_v, - "Predicate consumer data type must match its comparison basis"); static_assert(Index::Cols >= Basis::ValidCol && Value::Cols >= Basis::ValidCol, "Predicate operands must cover the complete valid region"); @@ -111,7 +187,7 @@ scatter(Global &destination, const Index &offsets, const Value &values, "Predicate destination element type must match values"); asm volatile( "BSTART.TLSU MSCATTER.MASK, %D[Type]\n" - "B.DATR CUBE_M32, Zero\n" + PTO_PREDICATE_LAYOUT_ASM "B.DIM zero, %c[Columns], ->LB0\n" "B.DIM zero, %c[Rows], ->LB1\n" "B.DIM zero, %c[PhysicalColumns], ->LB2\n" @@ -122,6 +198,7 @@ scatter(Global &destination, const Index &offsets, const Value &values, : [Values] "Tr"(values.data()), [Offsets] "Tr"(offsets.data()), [Predicate] "Tr"(active.Carrier), [Base] "r"(destination.data()), [Type] "i"(type_traits::TypeCode), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), [Columns] "i"(Basis::ValidCol), [Rows] "i"(Basis::ValidRow), [PhysicalColumns] "i"(Value::Cols) : "memory"); @@ -138,10 +215,244 @@ atomic_add(Global &destination, const Index &offsets, const Value &values, ew_mscatter_add_predicate( Basis::ValidRow, Basis::ValidCol, Value::Cols, type_traits::TypeCode, - 29, destination.data(), + Basis::BFractal == BLayout::CubeM16 ? 31 : 29, destination.data(), values.data(), offsets.data(), active.Carrier, Invert); } +template +constexpr void validate_selection() { + static_assert(Value::Loc == Location::Vec && + Value::BFractal == Basis::BFractal, + "Predicate selection requires matching Local CUBE layouts"); + static_assert(Value::ValidRow == Basis::ValidRow && + Value::ValidCol == Basis::ValidCol, + "Predicate selection valid shape must match"); + static_assert(std::is_same_v || + std::is_same_v || + std::is_same_v, + "Predicate selection supports FP32/S32/U32 values"); +} + +template +__attribute__((always_inline)) inline auto +subview(const Predicate &source) + -> Predicate> { + using ResultBasis = Tile; + Predicate result; + constexpr unsigned CellColumns = Basis::BFractal == BLayout::CubeM16 ? 8 : 4; + static_assert(Columns > 0 && Columns % CellColumns == 0, + "Predicate subview must cover complete CELL columns"); + static_assert(OffsetCells * CellColumns + Columns <= Basis::ValidCol, + "Predicate subview must fit the source valid region"); + static_assert(decltype(result)::Storage::TilesizeCode <= 10, + "Predicate subview capacity must be at most 64KB"); + asm volatile( + "BSTART.TLSU TMOV, U8\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[PhysicalColumns], ->LB2\n" + "B.IOT %[Source], mask=1111, last, ->%[Result]<%Z[Size]>\n" + "B.SUBVIEW 0, zero, %c[Offset], %c[Size]\n" + : [Result] "=&Tr"(result.Carrier) + : [Source] "Tr"(source.Carrier), [Offset] "i"(OffsetCells), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Basis::ValidRow), [Columns] "i"(Columns), + [PhysicalColumns] "i"(decltype(result)::StorageColumns), + [Size] "i"(decltype(result)::Storage::TilesizeCode)); + return result; +} + +template +__attribute__((always_inline)) inline auto +concatenate(const Predicate &left, const Predicate &right) + -> Predicate> { + static_assert(Left::BFractal == Right::BFractal && + Left::ValidRow == Right::ValidRow, + "Predicate assembly layouts and valid rows must match"); + static_assert(std::is_same_v && + std::is_same_v, + "Predicate assembly requires U8-basis logical results"); + static_assert(Left::ValidCol == Right::ValidCol && + Left::ValidCol == Predicate::StorageColumns && + Right::ValidCol == Predicate::StorageColumns, + "Predicate assembly requires equal complete writer regions"); + static_assert(Left::ValidRow == + (Left::BFractal == BLayout::CubeM16 ? 16 : 32), + "Predicate assembly requires full physical writer rows"); + static_assert(Predicate::Storage::TilesizeCode <= 10, + "Predicate assembly writer capacity must be at most 64KB"); + Predicate> result; + asm volatile( + "BSTART.TEPL TAND, U8\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[Columns], ->LB2\n" + "B.IOT %[Left], %[Left], mask=1111, last, ->%[Result]<%Z[Size]>\n" + "B.ASSEMBLE %c[Init], %c[Last], zero, 0, %c[WriterSize]\n" + : [Result] "=&Tr"(result.Carrier) + : [Left] "Tr"(left.Carrier), + [Layout] "i"(Left::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Left::ValidRow), [Columns] "i"(Left::ValidCol), + [Size] "i"(decltype(result)::Storage::TilesizeCode), + [Init] "i"(1), [Last] "i"(0), + [WriterSize] "i"(Predicate::Storage::TilesizeCode)); + asm volatile( + "BSTART.TEPL TAND, U8\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[Columns], ->LB2\n" + "B.IOT %[Right], %[Right], mask=1111, last\n" + "B.IOT %[Parent], mask=1111\n" + "B.ASSEMBLE %c[Init], %c[Last], zero, %c[Offset], %c[WriterSize]\n" + : + : [Parent] "Tr"(result.Carrier), [Right] "Tr"(right.Carrier), + [Layout] "i"(Left::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Right::ValidRow), [Columns] "i"(Right::ValidCol), + [Init] "i"(0), [Last] "i"(1), + [Offset] "i"(Predicate::Storage::StorageBytes / 128), + [WriterSize] "i"(Predicate::Storage::TilesizeCode)); + return result; +} + +template +__attribute__((always_inline)) inline auto +logical(const Predicate &left, const Predicate &right) + -> Predicate> { + static_assert(Operation == 6 || Operation == 7 || Operation == 8, + "Predicate logical operation must be AND, OR or XOR"); + static_assert(Left::BFractal == Right::BFractal && + Left::ValidRow == Right::ValidRow && + Left::ValidCol == Right::ValidCol, + "Predicate logical layouts and valid shapes must match"); + static_assert(Predicate::Storage::StorageBytes == + Predicate::Storage::StorageBytes && + Predicate::Storage::StorageBytes == + Predicate>::Storage::StorageBytes, + "Predicate logical physical shapes must match the result"); + Predicate> result; + asm volatile( + "BSTART.TEPL %c[Operation], U8\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[PhysicalColumns], ->LB2\n" + "B.IOT %[Left], %[Right], mask=1111, last, ->%[Result]<%Z[Size]>\n" + : [Result] "=Tr"(result.Carrier) + : [Left] "Tr"(left.Carrier), [Right] "Tr"(right.Carrier), + [Operation] "i"(Operation), + [Layout] "i"(Left::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Left::ValidRow), [Columns] "i"(Left::ValidCol), + [PhysicalColumns] "i"(decltype(result)::StorageColumns), + [Size] "i"(decltype(result)::Storage::TilesizeCode)); + return result; +} + +template +__attribute__((always_inline)) inline auto +logical_and(const Predicate &left, const Predicate &right) { + return logical<6>(left, right); +} + +template +__attribute__((always_inline)) inline auto +logical_or(const Predicate &left, const Predicate &right) { + return logical<7>(left, right); +} + +template +__attribute__((always_inline)) inline auto +logical_xor(const Predicate &left, const Predicate &right) { + return logical<8>(left, right); +} + +template +__attribute__((always_inline)) inline auto +logical_numeric(const Predicate &active, const Numeric &numeric) + -> LogicalBasis { + static_assert(Operation == 6 || Operation == 7 || Operation == 8, + "Predicate logical operation must be AND, OR or XOR"); + static_assert(Numeric::Loc == Location::Vec && + Numeric::BFractal == Basis::BFractal && + Numeric::ValidRow == Basis::ValidRow && + Numeric::ValidCol == Basis::ValidCol, + "Mixed logical layouts and valid shapes must match"); + static_assert(std::is_same_v || + std::is_same_v, + "Mixed logical input must be Numeric S8/U8"); + static_assert(Numeric::StorageBytes == Predicate::Storage::StorageBytes && + Numeric::StorageBytes == LogicalBasis::StorageBytes, + "Mixed logical physical shapes must match"); + LogicalBasis result; + asm volatile( + "BSTART.TEPL %c[Operation], U8\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[PhysicalColumns], ->LB2\n" + "B.IOT %[Predicate], %[Numeric], mask=1111, last, ->%[Result]<%Z[Size]>\n" + : [Result] "=Tr"(result.data()) + : [Predicate] "Tr"(active.Carrier), [Numeric] "Tr"(numeric.data()), + [Operation] "i"(Operation), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Basis::ValidRow), [Columns] "i"(Basis::ValidCol), + [PhysicalColumns] "i"(LogicalBasis::Cols), + [Size] "i"(LogicalBasis::TilesizeCode)); + return result; +} + +template +__attribute__((always_inline)) inline void +select(Value &destination, const Predicate &active, + const Value &when_true, const Value &when_false) { + validate_selection(); + asm volatile( + "BSTART.TEPL TSEL, %D[Type]\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[PhysicalColumns], ->LB2\n" + "B.IOT %[Predicate], %[True], mask=1111\n" + "B.IOT %[False], mask=1111, last, ->%[Result]<%Z[Size]>\n" + : [Result] "=Tr"(destination.data()) + : [Predicate] "Tr"(active.Carrier), [True] "Tr"(when_true.data()), + [False] "Tr"(when_false.data()), + [Type] "i"(type_traits::TypeCode), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Basis::ValidRow), [Columns] "i"(Basis::ValidCol), + [PhysicalColumns] "i"(Value::Cols), [Size] "i"(Value::TilesizeCode)); +} + +template +__attribute__((always_inline)) inline void +select_scalar(Value &destination, const Predicate &active, + const Value &when_true, typename Value::DType when_false) { + validate_selection(); + asm volatile( + "BSTART.TEPL TSELS, %D[Type]\n" + PTO_PREDICATE_LAYOUT_ASM + "B.DIM zero, %c[Columns], ->LB0\n" + "B.DIM zero, %c[Rows], ->LB1\n" + "B.DIM zero, %c[PhysicalColumns], ->LB2\n" + "B.IOT %[Predicate], %[True], mask=1111, last, ->%[Result]<%Z[Size]>\n" + "B.IOR [%[Scalar]], []\n" + : [Result] "=Tr"(destination.data()) + : [Predicate] "Tr"(active.Carrier), [True] "Tr"(when_true.data()), + [Scalar] "r"(when_false), + [Type] "i"(type_traits::TypeCode), + [Layout] "i"(Basis::BFractal == BLayout::CubeM16 ? 31 : 29), + [Rows] "i"(Basis::ValidRow), [Columns] "i"(Basis::ValidCol), + [PhysicalColumns] "i"(Value::Cols), [Size] "i"(Value::TilesizeCode)); +} + +#undef PTO_PREDICATE_LAYOUT_ASM + } } diff --git a/test/test_predicate_adapters.py b/test/test_predicate_adapters.py index 3359108..6c78e84 100644 --- a/test/test_predicate_adapters.py +++ b/test/test_predicate_adapters.py @@ -96,6 +96,138 @@ def test_builtin_wrong_arity(self): self.assertNotEqual(result.returncode, 0) self.assertIn("10", result.stderr) + def test_logical_selection_and_memory(self): + body = '''auto first = predicate::compare_scalar(values, threshold); +auto second = predicate::compare_scalar(values, threshold); +auto both = predicate::logical_and(first, second); +auto either = predicate::logical_or(first, second); +auto different = predicate::logical_xor(both, either); +static_assert(std::is_same_v); +predicate::select(values, different, values, values); +predicate::select_scalar(values, both, values, threshold); +predicate::scatter(destination, offsets, values, either); +predicate::atomic_add(destination, offsets, values, different);''' + obj = self.output / "logical.o" + result = self.compile(body, "-c", "-o", str(obj)) + self.assertEqual(result.returncode, 0, result.stderr) + text = subprocess.check_output([ + str(CXX.parent / "llvm-objdump"), "-d", + "--disassembler-options=no-tile-macros", str(obj)], text=True) + for operation in ("TAND", "TOR", "TXOR", "TSEL", "TSELS", + "MSCATTER.MASK", "MSCATTER.ADD"): + self.assertIn(operation, text) + self.assertNotIn("TCVT", text) + + def test_cross_basis_and_typed_views(self): + obj = self.output / "views.o" + result = self.compile('''VecTileM32 floats; +auto first = predicate::compare_scalar(values, threshold); +auto second = predicate::compare_scalar(floats, 1.0f); +auto combined = predicate::logical_and(first, second); +auto low = predicate::subview<0, 4>(combined); +auto high = predicate::subview<1, 4>(combined); +auto whole = predicate::concatenate(low, high); +predicate::scatter(destination, offsets, values, whole);''', + "-c", "-o", str(obj), columns=8) + self.assertEqual(result.returncode, 0, result.stderr) + text = subprocess.check_output([ + str(CXX.parent / "llvm-objdump"), "-d", + "--disassembler-options=no-tile-macros", str(obj)], text=True) + self.assertEqual(text.count("B.SUBVIEW"), 2, text) + self.assertEqual(text.count("B.ASSEMBLE"), 2, text) + source = self.output / "predicate.cpp" + assembly = self.output / "views.s" + result = subprocess.run(self.flags + ["-S", str(source), "-o", str(assembly)], + text=True, capture_output=True) + self.assertEqual(result.returncode, 0, result.stderr) + result = subprocess.run([ + str(CXX.parent / "llvm-mc"), "-triple=linx64v5", "-filetype=obj", + str(assembly), "-o", str(self.output / "views.roundtrip.o")], + text=True, capture_output=True) + self.assertEqual(result.returncode, 0, result.stderr) + + def test_numeric_is_not_logical_predicate(self): + result = self.compile('''auto active = predicate::compare_scalar(values, threshold); +auto invalid = predicate::logical_and(active, values);''', "-fsyntax-only") + self.assertNotEqual(result.returncode, 0) + self.assertIn("no matching function", result.stderr) + + def test_subview_bounds(self): + result = self.compile('''auto active = predicate::compare_scalar(values, threshold); +auto invalid = predicate::subview<1, 4>(active);''', "-fsyntax-only") + self.assertNotEqual(result.returncode, 0) + self.assertIn("must fit the source valid region", result.stderr) + + def test_mixed_result_is_numeric(self): + result = self.compile('''auto active = predicate::compare_scalar(values, threshold); +VecTileM32 numeric; +auto result = predicate::logical_numeric<6>(active, numeric); +static_assert(std::is_same_v); +predicate::scatter(destination, offsets, values, result);''', "-fsyntax-only") + self.assertNotEqual(result.returncode, 0) + self.assertIn("no matching function", result.stderr) + + def test_mixed_numeric_object(self): + result = self.compile('''auto active = predicate::compare_scalar(values, threshold); +VecTileM32 numeric; +auto result = predicate::logical_numeric<8>(active, numeric); +global_tensor> bytes(reinterpret_cast(destination.data())); +TSTORE(bytes, result);''', "-c", "-o", str(self.output / "mixed.o")) + self.assertEqual(result.returncode, 0, result.stderr) + + def test_logical_shape_mismatch(self): + result = self.compile('''auto first = predicate::compare_scalar(values, threshold); +VecTileM32 wider; +auto second = predicate::compare_scalar(wider, threshold); +auto invalid = predicate::logical_or(first, second);''', "-fsyntax-only") + self.assertNotEqual(result.returncode, 0) + self.assertIn("valid shapes must match", result.stderr) + + def test_cross_basis_selection(self): + result = self.compile('''auto active = predicate::compare_scalar(values, threshold); +VecTileM32 selected; +predicate::select(selected, active, selected, selected); +predicate::select_scalar(selected, active, selected, 0.0f);''', + "-c", "-o", str(self.output / "cross-basis.o")) + self.assertEqual(result.returncode, 0, result.stderr) + + def test_m16_memory_is_rejected(self): + source = self.output / "m16-memory.cpp" + source.write_text('''#include +using namespace pto; +using Values = Tile; +using Offsets = VecTileM32; +using Global = global_tensor>; +void test(Values &values, Offsets &offsets, Global &destination) { +auto active = predicate::compare_scalar(values, 0u); +predicate::scatter(destination, offsets, values, active); +} +''') + result = subprocess.run(self.flags + ["-fsyntax-only", str(source)], + text=True, capture_output=True) + self.assertNotEqual(result.returncode, 0) + self.assertIn("indexed memory currently requires CUBE_M32", result.stderr) + + def test_m16_consumers(self): + source = self.output / "m16.cpp" + source.write_text('''#include +using namespace pto; +using Values = Tile; +using Global = global_tensor>; +void test(Values &values, Global &destination, uint32_t threshold) { +auto first = predicate::compare_scalar(values, threshold); +auto combined = predicate::logical_xor(first, first); +predicate::select(values, combined, values, values); +predicate::select_scalar(values, combined, values, threshold); +auto view = predicate::subview<0, 8>(combined); +auto whole = predicate::concatenate(view, view); +} +''') + result = subprocess.run(self.flags + ["-c", str(source), "-o", + str(self.output / "m16.o")], + text=True, capture_output=True) + self.assertEqual(result.returncode, 0, result.stderr) + if __name__ == "__main__": unittest.main()