From f636a06df8f37c5680369de27aa3026ae63b2d05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ricardo=20Gro=C3=9F?= Date: Thu, 29 Oct 2026 20:55:23 +0100 Subject: [PATCH 1/2] fix: CMTK0003 reports assertions lost in nested combinators and through catches that cannot keep them A Task.WhenAny inside a Task.WhenAll or a Task.Run completes with the faulted assertion as its value, so awaiting the outer task saw no failure and the rule said nothing (#165). HandsOn now returns how deep the root holds the failure instead of a bool: 0 where its task faults with it (WhenAll, Run), one more for each task that completes with it as a value (WhenAny, WhenEach, an async lambda that returns it), one fewer for each await that unwraps one. DroppedAssertion treats an awaited, waited-for or converted combinator as observing only a failure at depth 0, so the same path reports await Task.WhenAll(Task.WhenAny(...)), await Task.Run(() => Task.WhenAny(...)), their Wait() and plain-Task forms, at any depth, and keeps await await Task.WhenAny(...) silent. Task.WaitAll of a WhenAny was already reported once, at the WhenAny's conversion to a plain Task. A catch around an awaited assertion in an async lambda counted as observing the failure whatever it did (#166). EnclosingAsyncFunction now stops only at a catch that keeps it: the await is in its try block, its type is or is a base of TypeKitAssertionException (resolved in the Testing assembly by its metadata name), and its handler can complete (control flow: endpoint reachable, or a return/break leaves it). A constant false filter leaves the handler unreachable, so control flow already reads it as never completing: an explicit filter check, seeded, was caught by nothing, and is not kept. catch { throw; }, a log and throw;, throw new ..., catch (OperationCanceledException) and when (false) now report. A filter that is no constant counts as keeping the failure: an Error reports only a failure lost whatever the filter answers. Both forms are written only with 2.3.0's Testing API, so decision 0118 covers them; no 2.2.0 build reports more. Input class: AIS, an assertion's task lost by a statement. Variants/DroppedTaskShapes gains 42 forms, all assertion-only: every combinator inside each other, three and four deep, arrays, Selects and async lambdas of WhenAny that return it, await it once, twice and three times; every catch kind (try/finally, rethrow, log and rethrow, throw another, unrelated type, false filter, nested tries, await in a handler or a finally) against silent controls (catch all, catch the assertion's exception, true filter, opaque filter, conditional rethrow, return, a later clause, nested tries that keep it); and Task.WaitAll/WaitAny of nested combinators. 1837 cases, from 830. IgnoredOutcomeAnalyzerTests holds the issues' snippets and a look-alike TypeKitAssertionException of another type. Seeded 11 ways, each caught by DroppedTaskVariantTests (failing cases): the producer's depth (192), WhenAny's depth (168), an async lambda's return (48), await unwrapping (48), a catch's try block only (48), the exception type (48), the rethrow check (144), its exit points (24), the exception's metadata name (48), the base chain (240), and an unreachable handler read as completing, which turns the when (false) rows red (24). README (CMTK0003) says so; claims T03-a-nested-when-any-is-reported, T03-a-catch-that-cannot-keep-the-failure-is-reported and T03-an-unknown-filter-counts-as-catching quote it. Closes #165 Closes #166 Defect-Class: AIS Found-By: OR Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- design/analyzers.md | 2 +- .../IgnoredOutcomeAnalyzer.cs | 101 ++++++++++++++---- src/CodoMetis.TypeKit.Analyzers/README.md | 8 ++ .../TypeKitSymbols.cs | 6 ++ .../IgnoredOutcomeAnalyzerTests.cs | 63 +++++++++++ .../Variants/DroppedTaskShapes.cs | 58 +++++++++- test/oracles/claims.json | 51 +++++++++ 8 files changed, 265 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 72e37d1..0641b5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ section is the GitHub release's notes. - `TraverseAsync` on `Option` and `Patch`: an asynchronous check of an optional input or a patch field, with a token form; nothing is called without a value ([Collect many results](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/docs/result.md#collect-many-results)). - `Choose()`, `Choose(selector)`, `SequenceAsync` and `TraverseAsync` read an `IAsyncEnumerable`, in order and without a list first; the token reaches the stream, nothing is read after the first error, and the enumerator is disposed however they stop ([Chain steps](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit/README.md#chain-steps)). - `Fault.Cause`: `Fault.Of(reason, detail, exception)` keeps the exception underneath for your log; equality and `ToString()` leave it out, and `ToHttpResult()` hands it to the host as `IFaultFeature.Cause` and `FaultProblem.Cause`, never into the problem ([What failed underneath](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit.AspNetCore/README.md#what-failed-underneath)). -- Testing: every assertion works on a `Task` or `ValueTask` of what it checks, `await service.FindAsync(id).ShouldBeSome()`; a task that throws fails the test with its own exception, and an assertion left without `await` is reported (CMTK0003): through `ConfigureAwait` or `WaitAsync`, in a branch of a `?:`, `??` or `switch`, and handed to `Task.WhenAll`, `Task.Run` or `Task.WhenEach` and dropped, or to `Task.WhenAny` or `Task.WaitAny` ([CodoMetis.TypeKit.Testing](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit.Testing/README.md)). +- Testing: every assertion works on a `Task` or `ValueTask` of what it checks, `await service.FindAsync(id).ShouldBeSome()`; a task that throws fails the test with its own exception, and an assertion left without `await` is reported (CMTK0003): through `ConfigureAwait` or `WaitAsync`, in a branch of a `?:`, `??` or `switch`, and handed to `Task.WhenAll`, `Task.Run` or `Task.WhenEach` and dropped, or to `Task.WhenAny` or `Task.WaitAny`, nested at any depth, or awaited in an `async` lambda whose `catch` throws again or can't catch it ([CodoMetis.TypeKit.Testing](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit.Testing/README.md)). - OpenAPI: `AddTypeKit(typeKit => typeKit.InlineValueObjects = true)` describes every value object in place by its underlying type's schema, in every position and in OpenAPI 3.1, 3.0 and 2.0, with no component of its own; closed sets keep theirs, and without the setting the document is unchanged ([Describe value objects as plain types](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/docs/aspnet-core.md#describe-value-objects-as-plain-types)). - Analyzers: CMTK0009 warns at `FirstOrNone`, `SingleOrNone` or `FirstOrError` on a query given a `Func` variable or a method group, which binds to the `IEnumerable` overload and reads every row; a lambda at the call or an `Expression` stays silent ([A condition that reads every row](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit.Analyzers/README.md#a-condition-that-reads-every-row)). - Docs: the guides, the package READMEs, the changelog and the API of every run-time package as one site with a menu and search, built from the latest release ([caffeinatedcoder.github.io/CodoMetis.TypeKit](https://caffeinatedcoder.github.io/CodoMetis.TypeKit/)). diff --git a/design/analyzers.md b/design/analyzers.md index 4ff1a57..a105409 100644 --- a/design/analyzers.md +++ b/design/analyzers.md @@ -33,7 +33,7 @@ exist. |---|---|---|---| | CMTK0001 | No `default`/`default(T)`/`new()`/`new T()` of a value object or a `[RequireCustomInitialization]` type. An operand of `==`/`!=`, or an argument of a call whose name has the word `Equal`, `Equals` or `Compare` or ends in `Be` (`ThrowIfEqual`, `Assert.NotEqual`, `ShouldNotBe`), is exempt: a guard check is the only defence against the defaults the rule cannot see. So is a `default` assigned to an `out` parameter of a method returning `bool`, the Try pattern. Where a sibling branch does not bind (same-project value objects), the target type is taken from the enclosing conditional, switch arm or collection | Error | shipped in 1.0.0 | | CMTK0002 | Type implements `IValue<>`/`IValidatedValue<,,>`, but the compilation does not reference `CodoMetis.TypeKit.Generators`, so it is never woven | Error | shipped in 1.0.0 | -| CMTK0003 | `Result`/`Option` a call returns, dropped by an expression statement, awaited or not, or a `Task` of one converted to a plain `Task` (`Task Cancel() => CancelAsync();`); `_ =` opts out. `Tap`, `TapAsync`, `TapError` and `TapErrorAsync` on a stored result or task are exempt, since they return their receiver, and so is a chain of them on one (`r.Tap(log).TapError(warn)`, `(await pending).Tap(log)`, `pending.WaitAsync(t).TapAsync(log)`), measured as an Error's false report before 2.0.0 was tagged; so is such a tap converted to a plain `Task` or waited for (`Task Log() => pending.TapAsync(log);`, `held.TapAsync(log).Wait()`), on a `Result` and on a `Task` of one, which an independent review found reported on 2026-10-14. Also a collection of results the statement made (an array, `ICollection` or `IReadOnlyCollection`, such as `await Task.WhenAll(ids.Select(orders.CancelAsync))`), and a method group of a `Task` method converted to a delegate returning `Task`; tasks held in variables are silent, and so is a lazy `IEnumerable`, where nothing has run, unless a call that was given no result returned it and the statement materialises it. Also `Wait` on a task the expression made, tasks of results the statement made converted to plain ones (`Task.WaitAll`, `WaitAny`), and a `foreach` or `await foreach` over results its header made whose body never reads the iteration variable; a variable named `_` opts out. From 2.3.0, an assertion of `CodoMetis.TypeKit.Testing` on a `Task` or `ValueTask` of what it checks, dropped by an expression statement without `await`: `service.FindAsync(id).ShouldBeNone();`, the lowered static call, through `?.`, the body of a lambda that returns nothing, through `ConfigureAwait` or `WaitAsync` of the `Task` or `Task` it returns (every overload), and at each branch of a `?:`, `??` or `switch` expression so passed through, judged by the path every dropped call takes (#152); handed to `Task.WhenAll`, `Task.Run` or `Task.WhenEach` whose task is dropped, or to `Task.WhenAny` or `Task.WaitAny` in any use, as arguments, an array, a collection expression, a `Select`, a query, a `ToList`, a method group or an `async` lambda that awaits it with no `catch` between, and a `WhenAny` of them converted to a plain `Task` or waited for, and a loop that never reads the assertions its header made (#157); `Task.WaitAll` throws, so it is silent. It fails the test only through its task. Known by the package's assembly, so a consumer's extension of the same name on a pending result is silent; awaited, held, returned or discarded with `_ =`, it is silent too, and so are held tasks combined. Also from 2.3.0, on tasks of results, forms 2.2.0 code can contain: a `Task.WhenAny` of tasks the statement made, in any use (#159; of held tasks it is silent, where 2.2.0 reported it awaited), `Wait()` on a `?:`, `??` or `switch` (#160), an array written in `Task.WaitAll` or `Task.WaitAny` (#161) and a dropped `Task.WhenEach` (#162). All held by `DroppedTaskVariantTests` over `Variants/DroppedTaskShapes`, 830 cases. CA1806 can only enforce this per method name, not per return type | Error | shipped in 1.0.0 at Warning; Error in 2.0.0 ([decision 50](adr/0050-warnings-cmtk0003-alone-becomes-error-2.md)), its 1.x gaps closed first; the dropped pending assertion in 2.3.0, a form only 2.3.0's API writes ([decision 118](adr/0118-an-error-may-learn-a-form-only-new-api-can-write.md)); the 2.2-writable forms #159 to #162 in 2.3.0, gaps found before the tag ([decision 119](adr/0119-a-gap-found-before-the-tag-ships-in-that-release.md)) | +| CMTK0003 | `Result`/`Option` a call returns, dropped by an expression statement, awaited or not, or a `Task` of one converted to a plain `Task` (`Task Cancel() => CancelAsync();`); `_ =` opts out. `Tap`, `TapAsync`, `TapError` and `TapErrorAsync` on a stored result or task are exempt, since they return their receiver, and so is a chain of them on one (`r.Tap(log).TapError(warn)`, `(await pending).Tap(log)`, `pending.WaitAsync(t).TapAsync(log)`), measured as an Error's false report before 2.0.0 was tagged; so is such a tap converted to a plain `Task` or waited for (`Task Log() => pending.TapAsync(log);`, `held.TapAsync(log).Wait()`), on a `Result` and on a `Task` of one, which an independent review found reported on 2026-10-14. Also a collection of results the statement made (an array, `ICollection` or `IReadOnlyCollection`, such as `await Task.WhenAll(ids.Select(orders.CancelAsync))`), and a method group of a `Task` method converted to a delegate returning `Task`; tasks held in variables are silent, and so is a lazy `IEnumerable`, where nothing has run, unless a call that was given no result returned it and the statement materialises it. Also `Wait` on a task the expression made, tasks of results the statement made converted to plain ones (`Task.WaitAll`, `WaitAny`), and a `foreach` or `await foreach` over results its header made whose body never reads the iteration variable; a variable named `_` opts out. From 2.3.0, an assertion of `CodoMetis.TypeKit.Testing` on a `Task` or `ValueTask` of what it checks, dropped by an expression statement without `await`: `service.FindAsync(id).ShouldBeNone();`, the lowered static call, through `?.`, the body of a lambda that returns nothing, through `ConfigureAwait` or `WaitAsync` of the `Task` or `Task` it returns (every overload), and at each branch of a `?:`, `??` or `switch` expression so passed through, judged by the path every dropped call takes (#152); handed to `Task.WhenAll`, `Task.Run` or `Task.WhenEach` whose task is dropped, or to `Task.WhenAny` or `Task.WaitAny` in any use, as arguments, an array, a collection expression, a `Select`, a query, a `ToList`, a method group or an `async` lambda that awaits it with no `catch` between that keeps its failure, and a `WhenAny` of them converted to a plain `Task` or waited for, and a loop that never reads the assertions its header made (#157); `Task.WaitAll` throws, so it is silent. A combinator inside another is judged by how deep its task holds the failure: `WhenAll` and `Run` fault with a task that faults, `WhenAny`, `WhenEach` and an `async` lambda that returns the task complete with it as a value, each `await` unwraps one, and an awaited, waited-for or converted combinator observes only a failure at depth 0, so `await Task.WhenAll(Task.WhenAny(…))` and `await Task.Run(() => Task.WhenAny(…))` report, at any depth, and `await await Task.WhenAny(…)` is silent (#165). A `catch` keeps the failure only when its `try` block throws it, its type is or is a base of `TypeKitAssertionException` (resolved in the Testing assembly), and its handler can complete, which a constant `false` filter leaves unreachable: `catch { throw; }`, a log and `throw;`, `throw new …`, `catch (OperationCanceledException)` and `when (false)` hand it on and report; a filter that is no constant counts as keeping it, since an Error reports only a failure lost whatever the filter answers (#166). It fails the test only through its task. Known by the package's assembly, so a consumer's extension of the same name on a pending result is silent; awaited, held, returned or discarded with `_ =`, it is silent too, and so are held tasks combined. Also from 2.3.0, on tasks of results, forms 2.2.0 code can contain: a `Task.WhenAny` of tasks the statement made, in any use (#159; of held tasks it is silent, where 2.2.0 reported it awaited), `Wait()` on a `?:`, `??` or `switch` (#160), an array written in `Task.WaitAll` or `Task.WaitAny` (#161) and a dropped `Task.WhenEach` (#162). All held by `DroppedTaskVariantTests` over `Variants/DroppedTaskShapes`, 1837 cases. CA1806 can only enforce this per method name, not per return type | Error | shipped in 1.0.0 at Warning; Error in 2.0.0 ([decision 50](adr/0050-warnings-cmtk0003-alone-becomes-error-2.md)), its 1.x gaps closed first; the dropped pending assertion in 2.3.0, a form only 2.3.0's API writes ([decision 118](adr/0118-an-error-may-learn-a-form-only-new-api-can-write.md)), its nested combinators and catches (#165, #166) included; the 2.2-writable forms #159 to #162 in 2.3.0, gaps found before the tag ([decision 119](adr/0119-a-gap-found-before-the-tag-ships-in-that-release.md)) | | CMTK0004 | `IValueObjectMaterializer<,>.Materialize`, `ValueObjectConverter<,>.Materialize` or a hand-written value object's public `Materialize`, called or referenced (expression trees included). The satellite's own call is compiled in the satellite; EF's compiled model is generated code, whose diagnostics are not reported (measured: reported, it is six errors in the smoke test's AOT consumer). `StoredJsonConverterFactory` goes through `IStoredJsonConverterSource`, not `Materialize` | Error | shipped in 1.0.0 | | CMTK0005 | `new T[n]`, `stackalloc T[n]`, `GC.AllocateUninitializedArray/AllocateArray` or `Array.Resize` of a no-default struct: every new slot starts as `default`. A constant length of zero (for `Array.Resize` and the `GC` methods too) or initial elements are silent | Warning | shipped in 1.0.0 | | CMTK0006 | Field or auto-property of a no-default struct type in a class that no initializer, `required` or constructor sets; a non-public parameterless constructor (EF, serializers) and a record's copy constructor are exempt. Measured 2026-09-28 on the repository's EF entities and request types: 12 of 12 reports on settable properties of a type with only the implicit constructor, each a real way to a default. `required`, the fix the message names where it compiles, works with EF's compiled model, precompiled queries and Native AOT (the smoke test rerun with it). A `[SetsRequiredMembers]` constructor must assign the `required` members it promises | Warning | shipped in 1.0.0 | diff --git a/src/CodoMetis.TypeKit.Analyzers/IgnoredOutcomeAnalyzer.cs b/src/CodoMetis.TypeKit.Analyzers/IgnoredOutcomeAnalyzer.cs index d0369b0..230f9ab 100644 --- a/src/CodoMetis.TypeKit.Analyzers/IgnoredOutcomeAnalyzer.cs +++ b/src/CodoMetis.TypeKit.Analyzers/IgnoredOutcomeAnalyzer.cs @@ -162,15 +162,17 @@ private static void Analyze(OperationAnalysisContext context, TypeKitSymbols sym /// Nor does Task.WhenAll(first.ShouldBeNone(), second.ShouldBeNone()); or Task.Run(() => /// pending.ShouldBeNone());, whose task faults when an assertion fails, or Task.WhenAny of them, /// awaited or not, whose task completes with the first to finish and leaves every failure unobserved, when - /// the statement made an assertion it hands on (). + /// the statement made an assertion it hands on (). Awaited, a WhenAll + /// or a Run observes only a failure it was handed as a fault: await Task.WhenAll(Task.WhenAny(…)) + /// completes with the faulted assertion as its value ('s depth). /// private static string? DroppedAssertion(IInvocationOperation invocation, bool awaited, Tasks tasks) { var method = invocation.TargetMethod; var dropped = IsPendingAssertion(method, tasks) ? !awaited - : tasks.PassesOnFailure(method) is { } faults && (!awaited || !faults) - && invocation.Arguments.Any(argument => MadeAssertions(argument.Value, tasks).Any()); + : tasks.PassesOnFailure(method) is { } faults + && invocation.Arguments.Any(argument => MadeAssertions(argument.Value, tasks).Any(made => !awaited || !faults || made.Depth > 0)); if (!dropped) return null; @@ -184,9 +186,10 @@ private static void Analyze(OperationAnalysisContext context, TypeKitSymbols sym /// given: new[] { a.ShouldBeNone(), b.ShouldBeNone() }, ids.Select(id => Find(id).ShouldBeNone()), /// pending.Select(OptionAssertions.ShouldBeNone), ids.Select(async id => await Find(id).ShouldBeNone()). /// An assertion assigned or discarded in it keeps its own task, and one awaited fails only the - /// async lambda it is in, which is handed on in its place. + /// async lambda it is in, which is handed on in its place. Each comes with the depth + /// finds it at. /// - private static IEnumerable<(IOperation Assertion, string Name)> MadeAssertions(IOperation value, Tasks tasks) + private static IEnumerable<(IOperation Assertion, string Name, int Depth)> MadeAssertions(IOperation value, Tasks tasks) { foreach (var operation in value.DescendantsAndSelf()) { @@ -197,25 +200,42 @@ private static void Analyze(OperationAnalysisContext context, TypeKitSymbols sym _ => null, }; - if (method is not null && IsPendingAssertion(method, tasks) && HandsOn(operation, value, tasks)) yield return (operation, method.Name); + if (method is not null && IsPendingAssertion(method, tasks) && HandsOn(operation, value, method, tasks) is { } depth) yield return (operation, method.Name, depth); } } - private static bool HandsOn(IOperation operation, IOperation root, Tasks tasks) + /// + /// How deep holds the failure of 's assertion, or + /// where it is not handed on, or observed on the way: 0 where root's task faults with + /// it, as a WhenAll's or a Run's does; one more for each task that completes with it as a value + /// instead, a WhenAny's or a WhenEach's, or an async lambda's that returns it; one fewer + /// for each await on the way that unwraps such a task. An await of the faulting task fails the + /// async lambda it is in, which is followed in its place. + /// + private static int? HandsOn(IOperation operation, IOperation root, IMethodSymbol assertion, Tasks tasks) { - for (var current = operation; !ReferenceEquals(current, root); current = current.Parent!) + var depth = 0; + + for (var current = operation; !ReferenceEquals(current, root);) { - // await pending.ShouldBeNone() in an async lambda fails the lambda's task: follow that instead, - // unless a catch on the way can swallow the failure. - if (current.Parent is IAwaitOperation) + var parent = current.Parent!; + + // await Task.WhenAny(pending.ShouldBeNone(), …) completes with the assertion's task, still unobserved; + // await pending.ShouldBeNone() fails the async lambda it is in, unless a catch on the way keeps the failure. + if (parent is IAwaitOperation) { - if (EnclosingAsyncFunction(current, root) is not { } function) return false; + if (depth > 0) + { + depth--; + current = parent; + } + else if (EnclosingAsyncFunction(current, root, assertion) is { } function) current = function; + else return null; - current = function; - if (ReferenceEquals(current, root)) return true; + continue; } - var handsOn = current.Parent switch + var handsOn = parent switch { IConversionOperation or IConditionalOperation or ICoalesceOperation or ISwitchExpressionArmOperation or ISwitchExpressionOperation => true, IArrayInitializerOperation or IArrayCreationOperation or ICollectionExpressionOperation or IArgumentOperation => true, @@ -225,20 +245,31 @@ IInvocationOperation call when ReferenceEquals(call.Instance, current) _ => false, }; - if (!handsOn) return false; + if (!handsOn) return null; + + // async () => Task.WhenAny(…) and Task.WhenAny(…) complete with the tasks they were given. + if (parent is IAnonymousFunctionOperation { Symbol.IsAsync: true } + || parent is IInvocationOperation combinator && !ReferenceEquals(combinator.Instance, current) && tasks.PassesOnFailure(combinator.TargetMethod) == false) + depth++; + + current = parent; } - return true; + return depth; } - /// The async lambda or anonymous method inside that fails, with no catch between them. - private static IOperation? EnclosingAsyncFunction(IOperation operation, IOperation root) + /// + /// The async lambda or anonymous method inside that + /// fails, with no catch between them that keeps the failure of (). + /// A catch keeps only what its try block throws, not what its handlers or its finally do. + /// + private static IOperation? EnclosingAsyncFunction(IOperation operation, IOperation root, IMethodSymbol assertion) { - for (var current = operation.Parent; current is not null && !ReferenceEquals(current, root); current = current.Parent) + for (IOperation? child = operation, current = operation.Parent; current is not null && !ReferenceEquals(current, root); child = current, current = current.Parent) { switch (current) { - case ITryOperation { Catches.Length: > 0 }: + case ITryOperation attempt when ReferenceEquals(child, attempt.Body) && attempt.Catches.Any(clause => Keeps(clause, assertion)): return null; case IAnonymousFunctionOperation function: return function.Symbol.IsAsync ? function : null; @@ -248,6 +279,32 @@ IInvocationOperation call when ReferenceEquals(call.Instance, current) return null; } + /// + /// Whether takes the failure of and keeps it: its type can + /// hold the TypeKitAssertionException the assertion throws, and its handler can complete, where + /// catch { throw; }, a log and throw;, or a throw new … hand the failure on. A constant + /// false filter leaves the handler unreachable, so it never completes either. A filter that is no + /// constant, when (Expected(e)), may take it and counts as taking it: an Error reports only a failure that + /// is lost whatever the filter answers. + /// + private static bool Keeps(ICatchClauseOperation clause, IMethodSymbol assertion) + { + if (TypeKitSymbols.AssertionException(assertion.ContainingAssembly) is { } thrown && !IsOrDerivesFrom(thrown, clause.ExceptionType)) return false; + + return clause.Handler is not { Syntax: BlockSyntax handler, SemanticModel: { } model } + || model.AnalyzeControlFlow(handler) is not { Succeeded: true, EndPointIsReachable: false, ExitPoints.IsEmpty: true }; + } + + private static bool IsOrDerivesFrom(ITypeSymbol? type, ITypeSymbol? ancestor) + { + for (; type is not null; type = type.BaseType) + { + if (SymbolEqualityComparer.Default.Equals(type, ancestor)) return true; + } + + return false; + } + /// /// An assertion of CodoMetis.TypeKit.Testing on a Task or ValueTask of the value it /// asserts on, await service.FindAsync(id).ShouldBeSome(): it awaits the value and fails the @@ -421,7 +478,7 @@ private static void AnalyzeWait(OperationAnalysisContext context, TypeKitSymbols if (method.Name != "WaitAny") return; foreach (var argument in wait.Arguments.Where(argument => argument.Parameter?.Ordinal == 0)) - foreach (var (assertion, name) in MadeAssertions(argument.Value, tasks)) + foreach (var (assertion, name, _) in MadeAssertions(argument.Value, tasks)) context.Report(Diagnostic.Create(Rule, assertion.Syntax.GetLocation(), "Task", name)); return; diff --git a/src/CodoMetis.TypeKit.Analyzers/README.md b/src/CodoMetis.TypeKit.Analyzers/README.md index 70b7ad7..b78e309 100644 --- a/src/CodoMetis.TypeKit.Analyzers/README.md +++ b/src/CodoMetis.TypeKit.Analyzers/README.md @@ -201,6 +201,14 @@ failure. It covers an array of them, a `Select`, a query, an `async` lambda that group such as `pending.Select(OptionAssertions.ShouldBeNone)`. `Task.WaitAll` throws the failures, so it is silent. So is a loop over assertions that reads each one. +Nesting doesn't hide them. A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you +await the outer task, because that task succeeds with the failed assertion as its value: +`await Task.WhenAll(Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone()))` passes while both assertions +fail. `await await Task.WhenAny(…)` throws the first failure, so it is silent. A `catch` around an awaited +assertion in an `async` lambda hides the failure only when it can keep it. A `catch` that throws again, one for +another exception, such as `catch (OperationCanceledException)`, and one with a `when (false)` filter are +reported. A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure. + **Collections.** A collection of results is dropped as silently as one result: ```csharp diff --git a/src/CodoMetis.TypeKit.Analyzers/TypeKitSymbols.cs b/src/CodoMetis.TypeKit.Analyzers/TypeKitSymbols.cs index 309bbe1..6952b38 100644 --- a/src/CodoMetis.TypeKit.Analyzers/TypeKitSymbols.cs +++ b/src/CodoMetis.TypeKit.Analyzers/TypeKitSymbols.cs @@ -114,10 +114,16 @@ private TypeKitSymbols( _computeRestriction = ComputeRestriction; } + /// The exception every assertion of CodoMetis.TypeKit.Testing fails with, by its metadata name in that assembly. + private const string AssertionExceptionName = "CodoMetis.TypeKit.Testing.TypeKitAssertionException"; + /// Whether is CodoMetis.TypeKit.Testing. public static bool IsTestingAssembly(IAssemblySymbol? assembly) => assembly is not null && string.Equals(assembly.Name, TestingAssembly, StringComparison.Ordinal); + /// The exception the assertions of , CodoMetis.TypeKit.Testing, fail with. + public static INamedTypeSymbol? AssertionException(IAssemblySymbol testing) => testing.GetTypeByMetadataName(AssertionExceptionName); + /// Whether the compilation references CodoMetis.TypeKit.Generators. public bool ReferencesGenerators { get; } diff --git a/test/CodoMetis.TypeKit.Analyzers.Tests/IgnoredOutcomeAnalyzerTests.cs b/test/CodoMetis.TypeKit.Analyzers.Tests/IgnoredOutcomeAnalyzerTests.cs index b9364d0..ca71528 100644 --- a/test/CodoMetis.TypeKit.Analyzers.Tests/IgnoredOutcomeAnalyzerTests.cs +++ b/test/CodoMetis.TypeKit.Analyzers.Tests/IgnoredOutcomeAnalyzerTests.cs @@ -191,6 +191,69 @@ public static void Run(Service service) """) .RunAsync(TestContext.Current.CancellationToken); + /// + /// A Task.WhenAny inside a Task.WhenAll or a Task.Run completes with the faulted assertion + /// as its value, so awaiting the outer task sees no failure (#165). Awaiting what the WhenAny completes + /// with sees it. + /// + [Fact] + public Task An_assertion_a_nested_WhenAny_completes_with_reports_and_awaited_twice_is_silent() => + TestWithAssertions( + """ + public static class Caller + { + public static async Task Check(Task> first, Task> second) + { + {|#0:await Task.WhenAll(Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone()))|}; + {|#1:await Task.Run(() => Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone()))|}; + {|#2:Task.WhenAll(Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone())).Wait()|}; + Func check = () => {|#3:Task.WhenAll(Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone()))|}; + await await Task.WhenAny(first.ShouldBeNone(), second.ShouldBeNone()); + await Task.WhenAll(Task.WhenAll(first.ShouldBeNone(), second.ShouldBeNone())); + } + } + """, + Cmtk0003(0, "Task", "WhenAll"), + Cmtk0003(1, "Task", "Run"), + Cmtk0003(2, "Task", "WhenAll"), + Cmtk0003(3, "Task", "WhenAll")) + .RunAsync(TestContext.Current.CancellationToken); + + /// + /// A catch around an awaited assertion in an async lambda keeps its failure only when it can: a + /// catch that throws again, one of a type that cannot hold a TypeKitAssertionException, a look-alike + /// of that name among them, or one whose filter is a constant false hands it on to the dropped task (#166). + /// + [Fact] + public Task An_assertion_a_catch_cannot_keep_reports_and_one_it_keeps_is_silent() => + TestWithAssertions( + """ + using System.Linq; + + public static class Lookalike + { + public sealed class TypeKitAssertionException : Exception { } + } + + public static class Caller + { + public static void Check(Task> pending) + { + {|#0:Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch { throw; } }))|}; + {|#1:Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch (OperationCanceledException) { } }))|}; + {|#2:Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch (Exception) when (false) { } }))|}; + {|#3:Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch (Lookalike.TypeKitAssertionException) { } }))|}; + Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch (TypeKitAssertionException) { } })); + Task.WhenAll(new[] { pending }.Select(async item => { try { await item.ShouldBeNone(); } catch (Exception exception) when (exception.Data.Count == 0) { } })); + } + } + """, + Cmtk0003(0, "Task", "WhenAll"), + Cmtk0003(1, "Task", "WhenAll"), + Cmtk0003(2, "Task", "WhenAll"), + Cmtk0003(3, "Task", "WhenAll")) + .RunAsync(TestContext.Current.CancellationToken); + [Fact] public void The_descriptor_is_the_published_contract() { diff --git a/test/CodoMetis.TypeKit.Analyzers.Tests/Variants/DroppedTaskShapes.cs b/test/CodoMetis.TypeKit.Analyzers.Tests/Variants/DroppedTaskShapes.cs index 6a9eb36..82ae846 100644 --- a/test/CodoMetis.TypeKit.Analyzers.Tests/Variants/DroppedTaskShapes.cs +++ b/test/CodoMetis.TypeKit.Analyzers.Tests/Variants/DroppedTaskShapes.cs @@ -45,7 +45,10 @@ public sealed record TaskPassThrough(string Name, string Suffix, bool KeepsTask /// Whether nothing is dropped in any use: the statement combines no task it made. /// What the expression is: a task, or what Task.WaitAny and Task.WaitAll return, which loses the tasks in every use. /// Whether an assertion's failure is seen all the same: Task.WaitAll throws it. -/// Whether only an assertion is written in the form, where a Result's would be a statement of its own. +/// +/// Whether only an assertion is written in the form: where a Result's would be a statement of its own, or a +/// combinator inside another, which only an assertion's failure passes through (#165). +/// /// Whether the sequence runs nothing until a loop reads it, so that dropping it drops nothing. public sealed record DropForm( string Name, @@ -118,7 +121,8 @@ public sealed record TaskUse(string Name, string Statement, bool? Drops, bool As /// /// Both calls go through the one path, so they agree on every form: #152 found the assertions alone silent /// through the pass-throughs of a plain Task and through the branches, #157 through the combinators, -/// and #159 a WhenAny of results dropped unawaited. +/// #159 a WhenAny of results dropped unawaited, #165 a WhenAny inside a WhenAll or a Run, +/// awaited, and #166 a catch that throws again, of another exception or filtered by false. /// /// public static class DroppedTaskShapes @@ -208,11 +212,61 @@ .. Assertion("ShouldHold", "FaultAssertions", "composed", "Fault.Refused", "Task new("Task.Run of a lambda", "Task.Run(() => {call}){w}", Combinator: "Run", AwaitDrops: false), new("Task.Run of an async lambda", "Task.Run(async () => await {call}){w}", Combinator: "Run", AwaitDrops: false), + // A combinator inside another (#165): WhenAll and Run fault with a task that faults, WhenAny completes with + // it as its value, at any depth, and each await unwraps one such task. + new("Task.WhenAll of a Task.WhenAll", "Task.WhenAll(Task.WhenAll({call}, {call})){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of a Task.WhenAny", "Task.WhenAll(Task.WhenAny({call}, {call})){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of a Task.Run", "Task.WhenAll(Task.Run(() => {call})){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAny of a Task.WhenAll", "Task.WhenAny(Task.WhenAll({call}, {call})){w}", Combinator: "WhenAny", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAny of a Task.WhenAny", "Task.WhenAny(Task.WhenAny({call}, {call})){w}", Combinator: "WhenAny", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAny of a Task.Run", "Task.WhenAny(Task.Run(() => {call})){w}", Combinator: "WhenAny", AwaitDrops: true, AssertionsOnly: true), + new("Task.Run of a Task.WhenAll", "Task.Run(() => Task.WhenAll({call}, {call})){w}", Combinator: "Run", AwaitDrops: false, AssertionsOnly: true), + new("Task.Run of a Task.WhenAny", "Task.Run(() => Task.WhenAny({call}, {call})){w}", Combinator: "Run", AwaitDrops: true, AssertionsOnly: true), + new("Task.Run of a Task.Run", "Task.Run(() => Task.Run(() => {call})){w}", Combinator: "Run", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of a Task.Run of a Task.WhenAll of a Task.WhenAll", "Task.WhenAll(Task.Run(() => Task.WhenAll(Task.WhenAll({call}, {call})))){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of a Task.Run of a Task.WhenAll of a Task.WhenAny", "Task.WhenAll(Task.Run(() => Task.WhenAll(Task.WhenAny({call}, {call})))){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of an array of Task.WhenAny", "Task.WhenAll(new[] { Task.WhenAny({call}, {call}), Task.WhenAny({call}, {call}) }){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of a Select of Task.WhenAny", "Task.WhenAll(Enumerable.Range(0, 2).Select(i => Task.WhenAny({call}, {call}))){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that return a Task.WhenAny", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => Task.WhenAny({call}, {call}))){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await a Task.WhenAny once", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => await Task.WhenAny({call}, {call}))){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await a Task.WhenAny twice", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => await await Task.WhenAny({call}, {call}))){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await a Task.WhenAny of a Task.WhenAny twice", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => await await Task.WhenAny(Task.WhenAny({call}, {call})))){w}", Combinator: "WhenAll", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await a Task.WhenAny of a Task.WhenAny three times", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => await await await Task.WhenAny(Task.WhenAny({call}, {call})))){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + + // A catch around the awaited assertion in an async lambda (#166) keeps the failure only when its type can + // hold a TypeKitAssertionException, no constant false filter refuses it and its handler can complete. A + // filter that is no constant may take it, and counts as taking it. + new("Task.WhenAll of async lambdas that try with only a finally", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } finally { GC.KeepAlive(i); } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch and rethrow", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch { throw; } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that log and rethrow", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception exception) { GC.KeepAlive(exception); throw; } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that throw another exception", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception exception) { throw new InvalidOperationException(\"wrapped\", exception); } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch an unrelated exception", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (OperationCanceledException) { } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch with a false filter", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception) when (false) { } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that rethrow inside a try that rethrows", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { try { await {call}; } catch { throw; } } catch (OperationCanceledException) { } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await in a catch's handler", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { GC.KeepAlive(i); } catch (Exception) { await {call}; } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that await in a finally", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { GC.KeepAlive(i); } catch (Exception) { } finally { await {call}; } })){w}", Combinator: "WhenAll", AwaitDrops: false, AssertionsOnly: true), + new("Task.Run of an async lambda that catches and rethrows", "Task.Run(async () => { try { await {call}; } catch { throw; } }){w}", Combinator: "Run", AwaitDrops: false, AssertionsOnly: true), + new("Task.WhenAny of async lambdas that catch and rethrow", "Task.WhenAny(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch { throw; } })){w}", Combinator: "WhenAny", AwaitDrops: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch everything", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch the assertion's exception", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (TypeKitAssertionException) { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch with a true filter", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception) when (true) { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch with a filter that is no constant", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception exception) when (exception.Data.Count == 0) { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that rethrow only sometimes", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception) { if (choose) throw; } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that return from the catch", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (Exception) { if (choose) return; throw; } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch in a later clause", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { await {call}; } catch (OperationCanceledException) { throw; } catch (Exception) { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that rethrow inside a try that catches", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { try { await {call}; } catch { throw; } } catch (Exception) { } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.WhenAll of async lambdas that catch inside a try that rethrows", "Task.WhenAll(Enumerable.Range(0, 2).Select(async i => { try { try { await {call}; } catch (Exception) { } } catch { throw; } })){w}", Combinator: "WhenAll", Silent: true, AssertionsOnly: true), + new("Task.Run of an async lambda that catches", "Task.Run(async () => { try { await {call}; } catch (Exception) { } }){w}", Combinator: "Run", Silent: true, AssertionsOnly: true), + new("Task.WaitAny of two calls", "Task.WaitAny([[{call}]], [[{call}]])", Value: FormValue.Index), new("Task.WaitAny of held tasks", "Task.WaitAny(held!, held!)", Value: FormValue.Index, Silent: true), new("Task.WaitAll of two calls", "Task.WaitAll([[{call}]], [[{call}]])", Value: FormValue.Nothing, AssertionsObserved: true), new("Task.WaitAny of an array", "Task.WaitAny(new[] { [[{call}]], [[{call}]] })", Value: FormValue.Index), new("Task.WaitAll of an array", "Task.WaitAll(new[] { [[{call}]], [[{call}]] })", Value: FormValue.Nothing, AssertionsObserved: true), + new("Task.WaitAll of a Task.WhenAll", "Task.WaitAll(Task.WhenAll([[{call}]], [[{call}]]))", Value: FormValue.Nothing, AssertionsObserved: true, AssertionsOnly: true), + // Task.WaitAll waits for the WhenAny as a plain Task, a task that never faults, reported where it converts. + new("Task.WaitAll of a Task.WhenAny", "Task.WaitAll([[Task.WhenAny({call}, {call})]])", Combinator: "WhenAny", Value: FormValue.Nothing, AssertionsOnly: true), + new("Task.WaitAny of a Task.WhenAll", "Task.WaitAny(Task.WhenAll([[{call}]], [[{call}]]))", Value: FormValue.Index, AssertionsOnly: true), new("Task.WhenAll of a query", "Task.WhenAll(from i in Enumerable.Range(0, 2) select {call}){w}", Combinator: "WhenAll", AwaitDrops: false), new("Task.WhenAll of a list", "Task.WhenAll(Enumerable.Range(0, 2).Select(i => {call}).ToList()){w}", Combinator: "WhenAll", AwaitDrops: false), diff --git a/test/oracles/claims.json b/test/oracles/claims.json index df6f69b..d3a2772 100644 --- a/test/oracles/claims.json +++ b/test/oracles/claims.json @@ -4820,6 +4820,57 @@ "DroppedTaskVariantTests" ] }, + { + "id": "T03-a-nested-when-any-is-reported", + "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", + "quote": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", + "claim": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", + "oracle": "claim", + "surfaces": [ + "OptionAssertions.*", + "ResultAssertions.*", + "PatchAssertions.*", + "FaultAssertions.*" + ], + "heldBy": [ + "DroppedTaskVariantTests", + "IgnoredOutcomeAnalyzerTests" + ] + }, + { + "id": "T03-a-catch-that-cannot-keep-the-failure-is-reported", + "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", + "quote": "A `catch` that throws again, one for another exception, such as `catch (OperationCanceledException)`, and one with a `when (false)` filter are reported.", + "claim": "A `catch` that throws again, one for another exception, such as `catch (OperationCanceledException)`, and one with a `when (false)` filter are reported.", + "oracle": "claim", + "surfaces": [ + "OptionAssertions.*", + "ResultAssertions.*", + "PatchAssertions.*", + "FaultAssertions.*" + ], + "heldBy": [ + "DroppedTaskVariantTests", + "IgnoredOutcomeAnalyzerTests" + ] + }, + { + "id": "T03-an-unknown-filter-counts-as-catching", + "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", + "quote": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", + "claim": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", + "oracle": "claim", + "surfaces": [ + "OptionAssertions.*", + "ResultAssertions.*", + "PatchAssertions.*", + "FaultAssertions.*" + ], + "heldBy": [ + "DroppedTaskVariantTests", + "IgnoredOutcomeAnalyzerTests" + ] + }, { "id": "T02-when-any-of-results-is-reported", "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", From d90275eccc9799909609c878a374fbff17ce8e7e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ricardo=20Gro=C3=9F?= Date: Thu, 29 Oct 2026 21:00:07 +0100 Subject: [PATCH 2/2] test: the T03 claims sort after T02, as the catalogue's ids must Co-Authored-By: Claude Opus 5.5 --- test/oracles/claims.json | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/test/oracles/claims.json b/test/oracles/claims.json index d3a2772..47a89c5 100644 --- a/test/oracles/claims.json +++ b/test/oracles/claims.json @@ -4821,20 +4821,17 @@ ] }, { - "id": "T03-a-nested-when-any-is-reported", + "id": "T02-when-any-of-results-is-reported", "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", - "quote": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", - "claim": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", + "quote": "A `WhenAny` of tasks the statement made is reported whether you drop it, await it, wait for it or return it as a plain `Task`.", + "claim": "A `WhenAny` of tasks the statement made is reported whether you drop it, await it, wait for it or return it as a plain `Task`.", "oracle": "claim", "surfaces": [ - "OptionAssertions.*", - "ResultAssertions.*", - "PatchAssertions.*", - "FaultAssertions.*" + "Result.*" ], "heldBy": [ - "DroppedTaskVariantTests", - "IgnoredOutcomeAnalyzerTests" + "IgnoredOutcomeAnalyzerTests", + "DroppedTaskVariantTests" ] }, { @@ -4855,10 +4852,10 @@ ] }, { - "id": "T03-an-unknown-filter-counts-as-catching", + "id": "T03-a-nested-when-any-is-reported", "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", - "quote": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", - "claim": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", + "quote": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", + "claim": "A `Task.WhenAny` inside a `Task.WhenAll` or a `Task.Run` is reported even when you await the outer task, because that task succeeds with the failed assertion as its value", "oracle": "claim", "surfaces": [ "OptionAssertions.*", @@ -4872,17 +4869,20 @@ ] }, { - "id": "T02-when-any-of-results-is-reported", + "id": "T03-an-unknown-filter-counts-as-catching", "source": "src/CodoMetis.TypeKit.Analyzers/README.md#cmtk0003", - "quote": "A `WhenAny` of tasks the statement made is reported whether you drop it, await it, wait for it or return it as a plain `Task`.", - "claim": "A `WhenAny` of tasks the statement made is reported whether you drop it, await it, wait for it or return it as a plain `Task`.", + "quote": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", + "claim": "A filter the rule can't work out, such as `when (Expected(e))`, counts as catching the failure.", "oracle": "claim", "surfaces": [ - "Result.*" + "OptionAssertions.*", + "ResultAssertions.*", + "PatchAssertions.*", + "FaultAssertions.*" ], "heldBy": [ - "IgnoredOutcomeAnalyzerTests", - "DroppedTaskVariantTests" + "DroppedTaskVariantTests", + "IgnoredOutcomeAnalyzerTests" ] }, {