Factorization optimization: Emit factors incrementally and add optional small-degree screening

libs/math/docs/polynomial_factorization.md

@@ -114,6 +114,40 @@ The factor that triggers the stop is included in `result.factors`, and `result.c
 `complete == true`. Square-free and distinct-degree factorization also expose staged callbacks, allowing a caller to
 stop at the earliest useful stage rather than computing a complete irreducible factorization.
 
+Complete factorization splits each distinct-degree group and invokes its irreducible-factor callbacks as soon as the
+group is discovered. A callback stop also stops distinct-degree discovery, so later groups and square-free parts are
+not processed. Factors retain their square-free multiplicities and the same group/splitting order. The shared
+generator is consumed in the same order, including when callbacks use it, and callback or generator exceptions
+propagate to the caller.
+
+By default, each nonlinear square-free part prepares the initial Frobenius power and full baby-step table before
+discovering groups. To screen small degrees first, set a numeric limit on the existing arithmetic options:
+
+```cpp
+math::polynomial_arithmetic::polynomial_context_options options;
+options.distinct_degree_screening_limit = 1; // Screen linear factors before building the full baby table.
+math::polynomial_arithmetic::polynomial_context<backend_type> arithmetic_context(backend_type{}, options);
+```
+
+The default limit is zero, preserving the unscreened baseline. Positive limits are capped at the Kaltofen-Shoup
+degree-block size. Screening computes `gcd(remaining, X^(Q^d) - X)` in increasing degrees `d`, removing each emitted
+group before advancing. These are equal-degree groups; their combined norm must not be used to classify every
+constituent factor. Complete factorization splits them individually and preserves square-free multiplicities, so
+norm recovery still handles even powers without rejecting their individually unsuitable factors.
+
+Screening reuses the initial `X^Q mod B` and subsequent Frobenius powers, all computed modulo the original
+square-free part B. A callback stop avoids constructing the remaining baby table and its block-composition cache.
+It still pays for square-free decomposition, the initial power, and the initial Frobenius composition cache. If
+screening continues, its powers become the prefix of the full baby table and only missing steps are computed.
+The blocked search retains its original alignment; its first block may revisit screened degrees, whose factors
+have already been removed. This adds GCD/exact-division work relative to the baseline and should be measured on
+both rejected and suitable inputs before choosing a limit. Passing a bounded screen does not establish that norm
+recovery will succeed.
+
+Splitting and callback work run while the active Frobenius precomputation remains alive, which can increase peak
+memory. Splitting copies the emitted group because distinct-degree refinement or screening still needs its stored
+polynomial after the callback returns. No default screening bound is selected from the behavior of one candidate.
+
 ## Preconditions and current restrictions
 
 * Inputs are nonempty coefficient polynomials. Algorithms canonicalize trailing zero coefficients and return monic

libs/math/include/nil/crypto3/math/polynomial/backends/polynomial_backend.hpp

@@ -52,6 +52,11 @@ namespace nil::crypto3::math::polynomial_arithmetic {
         // has at most degree(divisor) coefficients. The limit must be positive; the default permits the algorithm's
         // square-root block size.
         std::size_t modular_composition_cached_power_limit = std::numeric_limits<std::size_t>::max();
+
+        // Examine initial irreducible-factor degrees before constructing the full Kaltofen-Shoup baby table.
+        // Zero keeps the unscreened path. The limit is capped at the degree-block size so the computed Frobenius
+        // powers can be reused as the start of that table if factorization continues.
+        std::size_t distinct_degree_screening_limit = 0;
     };
 
     /**

libs/math/include/nil/crypto3/math/polynomial/factorization/complete_factorization.hpp

@@ -51,8 +51,11 @@ namespace nil::crypto3::math {
      * generator is shared by all Cantor-Zassenhaus groups and must return independent uniformly distributed
      * coefficient-field elements.
      *
-     * After each irreducible factor is appended to the result, factor_callback may request an early stop. A stopped
-     * result includes that factor and has complete set to false. Zero and constant inputs produce no factors and
+     * Each distinct-degree group is split as soon as it is discovered, before searching for later groups. After each
+     * irreducible factor is appended to the result, factor_callback may request an early stop. This stops both group
+     * splitting and distinct-degree discovery; the result includes that factor and has complete set to false. The
+     * default path constructs the Kaltofen-Shoup baby table before discovering groups. Configured small-degree
+     * screening can deliver factors before that table is built. Zero and constant inputs produce no factors and
      * preserve their scalar value as leading_coefficient.
      *
      * @throws std::invalid_argument under the restrictions documented by the component stages, including when the
@@ -77,22 +80,23 @@ namespace nil::crypto3::math {
             const std::size_t block_size = detail::kaltofen_shoup_block_size(square_free_factor.polynomial.size() - 1);
 
             std::vector<distinct_degree_factor<polynomial_type>> degree_groups;
-            detail::kaltofen_shoup_factor_monic_square_free(degree_groups, std::move(square_free_factor.polynomial),
-                                                            block_size, arithmetic_context,
-                                                            [](const distinct_degree_factor<polynomial_type> &) {
-                                                                return factorization_control::continue_factorization;
-                                                            });
-
-            for (auto &degree_group : degree_groups) {
-                const factorization_control control = detail::factor_distinct_degree_group<Backend>(
-                    std::move(degree_group), arithmetic_context, generator, [&](polynomial_type &&factor) {
-                        result.factors.push_back({std::move(factor), multiplicity});
-                        return factor_callback(result.factors.back());
-                    });
-                if (control == factorization_control::stop_factorization) {
-                    result.complete = false;
-                    return result;
-                }
+            const auto control = detail::kaltofen_shoup_factor_monic_square_free(
+                degree_groups, std::move(square_free_factor.polynomial), block_size, arithmetic_context,
+                [&](const distinct_degree_factor<polynomial_type> &degree_group) {
+                    // Split each newly discovered degree group now and report its irreducible factors immediately.
+                    // This lets the caller stop before Kaltofen-Shoup searches for later degree groups.
+                    // The splitter takes its group by value, so passing degree_group intentionally copies it.
+                    // Splitting consumes that copy; Kaltofen-Shoup still needs the original polynomial for exact
+                    // division if this callback continues.
+                    return detail::factor_distinct_degree_group<Backend>(
+                        degree_group, arithmetic_context, generator, [&](polynomial_type &&factor) {
+                            result.factors.push_back({std::move(factor), multiplicity});
+                            return factor_callback(result.factors.back());
+                        });
+                });
+            if (control == factorization_control::stop_factorization) {
+                result.complete = false;
+                return result;
             }
         }
         return result;

libs/math/include/nil/crypto3/math/polynomial/factorization/kaltofen_shoup_distinct_degree_factorization.hpp

@@ -74,10 +74,12 @@ namespace nil::crypto3::math::detail {
      * Reusing that composition advances consecutive giant steps with one modular composition per block.
      *
      * Constructing the baby-step table performs b Frobenius maps and stores b + 1 residues, using
-     * O(b * degree(B)) field elements. Construction also builds the existing modular-composition precomputation for
-     * h[b].
+     * O(b * degree(B)) field elements. A prefix already computed by small-degree screening can be moved into the
+     * table; only missing steps are computed. Construction also builds the existing modular-composition
+     * precomputation for h[b].
      *
-     * @throws std::invalid_argument if block_size is zero.
+     * @throws std::invalid_argument if block_size is zero or the supplied prefix extends beyond h[b].
+     * @pre a nonempty initial_baby_steps contains h[0], ..., h[s] for this same frobenius_context, with s <= b.
      */
     template<SupportsDivrem Backend>
     class kaltofen_shoup_frobenius_precomputation {
@@ -88,8 +90,10 @@ namespace nil::crypto3::math::detail {
 
         kaltofen_shoup_frobenius_precomputation(
             std::size_t block_size, const polynomial_frobenius_context<backend_type> &frobenius_context,
-            polynomial_arithmetic::polynomial_context<backend_type> &arithmetic_context) :
-            block_size_(block_size), baby_steps_(make_baby_steps(block_size_, frobenius_context, arithmetic_context)),
+            polynomial_arithmetic::polynomial_context<backend_type> &arithmetic_context,
+            std::vector<polynomial_type> initial_baby_steps = {}) :
+            block_size_(block_size), baby_steps_(make_baby_steps(block_size_, frobenius_context, arithmetic_context,
+                                                                 std::move(initial_baby_steps))),
             block_frobenius_precomputation_(baby_steps_.back(),
                                             // A residue modulo degree-d polynomial B has at most d coefficients. The
                                             // composition precomputation requires a positive limit for constant B.
@@ -122,22 +126,27 @@ namespace nil::crypto3::math::detail {
     private:
         static std::vector<polynomial_type>
             make_baby_steps(std::size_t block_size, const polynomial_frobenius_context<backend_type> &frobenius_context,
-                            polynomial_arithmetic::polynomial_context<backend_type> &arithmetic_context) {
+                            polynomial_arithmetic::polynomial_context<backend_type> &arithmetic_context,
+                            std::vector<polynomial_type> baby_steps) {
             if (block_size == 0) {
                 throw std::invalid_argument("the Kaltofen-Shoup block size must be positive");
             }
 
-            std::vector<polynomial_type> baby_steps;
+            if (!baby_steps.empty() && baby_steps.size() - 1 > block_size) {
+                throw std::invalid_argument("the supplied Frobenius prefix exceeds the Kaltofen-Shoup block size");
+            }
             baby_steps.reserve(block_size + 1);
-            polynomial_type reduced_x = {value_type {}, value_type::one()};
-            const auto &divisor_context = frobenius_context.divisor_context();
-            if (divisor_context.degree() == 0) {
-                reduced_x.assign(1, value_type {});
-            } else if (reduced_x.size() >= divisor_context.divisor().size()) {
-                remainder(reduced_x, reduced_x, divisor_context, arithmetic_context);
+            if (baby_steps.empty()) {
+                polynomial_type reduced_x = {value_type {}, value_type::one()};
+                const auto &divisor_context = frobenius_context.divisor_context();
+                if (divisor_context.degree() == 0) {
+                    reduced_x.assign(1, value_type {});
+                } else if (reduced_x.size() >= divisor_context.divisor().size()) {
+                    remainder(reduced_x, reduced_x, divisor_context, arithmetic_context);
+                }
+                baby_steps.emplace_back(std::move(reduced_x));
             }
-            baby_steps.emplace_back(std::move(reduced_x));
-            for (std::size_t index = 1; index <= block_size; ++index) {
+            for (std::size_t index = baby_steps.size(); index <= block_size; ++index) {
                 polynomial_type next_step;
                 frobenius_map(next_step, baby_steps.back(), frobenius_context, arithmetic_context);
                 baby_steps.emplace_back(std::move(next_step));
@@ -270,6 +279,8 @@ namespace nil::crypto3::math::detail {
      * For each consecutive degree block, the algorithm computes one coarse GCD, removes that block from the
      * unclassified polynomial, and immediately fine-splits it. Interleaving the coarse and fine phases avoids storing
      * a giant-step polynomial for every block while preserving increasing degree order and immediate callback stops.
+     * Optional small-degree screening uses successive Frobenius differences before building the full baby table.
+     * The powers and divisor context remain relative to the original input even after screened factors are removed.
      *
      * Once the smallest unprocessed factor degree is greater than half the degree of the unclassified polynomial,
      * that polynomial is either constant or irreducible. Otherwise it would contain at least two factors of at least
@@ -287,6 +298,7 @@ namespace nil::crypto3::math::detail {
         typename Backend::polynomial_type input, std::size_t block_size,
         polynomial_arithmetic::polynomial_context<Backend> &arithmetic_context, FactorCallback &&factor_callback) {
         using polynomial_type = typename Backend::polynomial_type;
+        using value_type = typename polynomial_type::value_type;
 
         if (block_size == 0) {
             throw std::invalid_argument("the Kaltofen-Shoup block size must be positive");
@@ -298,11 +310,53 @@ namespace nil::crypto3::math::detail {
         }
 
         polynomial_frobenius_context<Backend> frobenius_context(input, arithmetic_context);
-        kaltofen_shoup_frobenius_precomputation<Backend> precomputation(block_size, frobenius_context,
-                                                                        arithmetic_context);
-        polynomial_type giant_step = precomputation.baby_step(block_size);
         polynomial_type unclassified(std::move(input));
+        std::vector<polynomial_type> initial_baby_steps;
+        const auto screening_limit = std::min(arithmetic_context.options().distinct_degree_screening_limit, block_size);
+        if (screening_limit != 0) {
+            const polynomial_type x = {value_type {}, value_type::one()};
+            initial_baby_steps.push_back(x);
+            std::size_t degree = 1;
+            for (; degree <= screening_limit && degree <= (unclassified.size() - 1) / 2; ++degree) {
+                if (degree == 1) {
+                    // The initial X^Q mod B is already available; do not repeat its exponentiation or composition.
+                    initial_baby_steps.push_back(frobenius_context.x_to_field_order());
+                } else {
+                    polynomial_type next_power;
+                    frobenius_map(next_power, initial_baby_steps.back(), frobenius_context, arithmetic_context);
+                    initial_baby_steps.push_back(std::move(next_power));
+                }
+                polynomial_type difference, factor;
+                subtraction(difference, initial_baby_steps.back(), x);
+                gcd(factor, unclassified, difference, arithmetic_context);
+                if (factor.size() > 1) {
+                    // All lower degrees have been removed, so this is an exact-degree group, not necessarily one
+                    // irreducible factor. Let the existing callback split it and apply multiplicity-aware recovery.
+                    output.push_back({std::move(factor), degree});
+                    if (factor_callback(output.back()) == factorization_control::stop_factorization)
+                        return factorization_control::stop_factorization;
+                    polynomial_type quotient;
+                    factorization_exact_quotient(quotient, unclassified, output.back().polynomial, arithmetic_context);
+                    unclassified = std::move(quotient);
+                }
+            }
+            // Once the remaining degree is less than twice the next untested degree, at most one factor remains.
+            if (degree > (unclassified.size() - 1) / 2) {
+                if (unclassified.size() > 1) {
+                    const auto remaining_degree = unclassified.size() - 1;
+                    output.push_back({std::move(unclassified), remaining_degree});
+                    return factor_callback(output.back());
+                }
+                return factorization_control::continue_factorization;
+            }
+        }
+
+        kaltofen_shoup_frobenius_precomputation<Backend> precomputation(
+            block_size, frobenius_context, arithmetic_context, std::move(initial_baby_steps));
+        polynomial_type giant_step = precomputation.baby_step(block_size);
 
+        // Keep the original degree-block alignment. The first block may revisit screened degrees, but their factors
+        // have been removed from unclassified and therefore cannot be emitted twice.
         std::size_t first_factor_degree = 1;
         while (first_factor_degree <= (unclassified.size() - 1) / 2) {
             const std::size_t maximum_degree_to_test = (unclassified.size() - 1) / 2;