[awsmadi]

Continues #717, which was merged at 567c0d7. These are the commits made on that branch afterwards and so
not part of it. Rebased onto current main; the tree differs from the pre-rebase branch only in the
lockfiles ef17f36 patched.

Same class of defect as #717: a rule that answers compliance for a check that never ran, or a run that
exits non-zero and names no finding.

One CI job is expected to be red, and it is not this branch's doing.
aws-guard-rules-registry-integration-tests-* fails because a change here — failing a run when an
expectation names a rule that produces no verdict — is correct and the registry has 30 such expectations, all
in rules/aws/aws_cloudformation, naming 11 rules their file never defines. Measured: exactly 1 of 49
registry directories changes its exit code, and stdout is byte-identical over the whole corpus, 9,156 lines
either way. The fix is aws-cloudformation/aws-guard-rules-registry#288, which takes
cfn-guard test -d rules on that repository from exit 1 to exit 0 against this branch's binary. Both registry
jobs are also now pinned to a SHA instead of tracking main, so a commit in another repository can no longer
turn this one's CI red without a change here; read that pin as a lock on a known state rather than a green
one, and move it forward once #288 lands.

A typo could delete a rule and report compliance

extract_message searched all remaining input for the closing >>, so one forgotten tag consumed every
rule up to the next rule's tag as message text and those rules ceased to exist. A file whose second rule
the template violated reported PASS at exit 0 — no finding, no diagnostic on any channel, and a parse
tree holding only the first rule with the rest of the file inside its custom message.

A message is now delimited by the grammar the language actually uses, rather than by a bound on the search.
A >> on the opening line closes the message; otherwise the message ends at the first later line whose
trimmed text is exactly >>; otherwise it is the error this code already raised for a missing tag.

Two earlier attempts at this each bounded the search, and each was wrong. They are worth stating because
the shape of the mistake is the same both times. Stopping at the first line whose first token is } or rule
finds a forgotten tag whose next >> is in a later rule, but when the next >> belongs to the next clause of
the same block no such line sits between the tags, so that clause was still swallowed at exit 0 -- in a rule
body, in a type block and in a when block. It also rejected a legitimate body quoting example JSON, because
such a body has a line starting }. Stopping at the next << fixed all of those and broke a third case: with
no second << in the file the search ran on to a >> inside a comment and closed the message there, so

rule one {
    Resources.One.Type == "AWS::S3::Bucket" << closing tag forgotten
}
rule two {
    Resources.One.Properties.Encrypted == true
    # see the runbook for escalation >>
}
rule three {
    Resources.One.Properties.Public == true
}

parsed as rules one and three, with rule two absent from the parse tree, and exited 0 against a template rule
two violates. Taking the minimum of the two bounds does not work either: a comment carrying >> at the end of
the same block is silent under both, since neither a } line nor a second << sits between the tags.

Each of those bounds inferred document structure from a token that may legitimately appear inside a message
body. A terminator line infers nothing -- it is what a closing tag looks like in 231 of the 233 messages that
exist. The census is the whole basis for the design: of the 233 messages in the AWS rule registry and this
repository's fixtures, 231 are block form with the closing >> alone on its line, 2 are inline with both tags
on one line, nothing follows >> on its line in any of them, and there are no other shapes.

Two behaviour changes fall out. The JSON-quoting body now parses, because a brace in a body is text, so this
removes a limitation rather than documenting one. And a block body whose closing >> shares its line with body
text, as in << Violation: X then Fix: Y >>, is now a parse error. That is a narrowing, and it cannot be
avoided: the swallowed clause in the example above is itself a line ending in >>, so anything that accepts
the one accepts the other. No message in the corpus is written that way. It is asserted as a rejection so it
is not reopened by loosening the terminator.

All four broken shapes exit 5 now, for one reason rather than four. Measured against the previous bound over
2,494 fixture pairs and 276 rules files: zero exit-code, stdout and stderr differences, and byte-identical
parse trees, so all 233 messages parse exactly as before. Regression tests cover the four broken shapes, both
accepted forms, the empty message, and the rejectedthree shapes.

Five more readings that were not what the author wrote

• A leading space changed the query part. [x] gave AllIndices and [ x ] gave AllValues, because
  the named branch of all_indices did not skip whitespace. Two spaces were then the difference between a
  working rule and an unevaluatable one: Tags.%k[x] == "prod" passed at exit 0 while Tags.%k[ x ] bailed
  at exit 19, since the %var interpolation arm accepts one query part and not the other.
• when was a tag in front of a cut. It matched the first four characters of whenCreated, the
  whitespace requirement after it failed, and the cut made that unrecoverable — so any clause whose first
  identifier began when or WHEN was a parse failure. rule whenever { ... } defined fine and a
  whenever reference did not: a rule you could declare and never call.
• A name assigned twice in one scope resolved by kind precedence, not order. let v = 1 then
  let v = 999 failed a rule that the reverse order passed, and two assignments of different kinds
  ignored their order entirely, because literals, queries and function calls go into separate maps consulted
  in a fixed order. Rejected now in both scopes. count.guard in this repository was a live instance: it
  bound res twice and asserted %res == 3 after each, so the first assertion tested the second count —
  and would have failed had it been live, because there are five properties and three collection entries.
  Both counts are now named and asserted.
• keys was reserved for every clause rather than for key filters. A cut after the keyword stopped
  [ keys EXISTS ] from reaching the branch that parses it. The identical clause parsed one slot later,
  which is what showed nothing ambiguous forced the rejection.
• in and the seven IS_* operators were plain tags. Size IS_INTEGER parsed as IS_INT plus a
  reference to a rule called EGER — loud with no such rule, PASS at exit 0 with one. The same guard
  true, false and null already had.

Three smaller ones: i64::MIN was the single integer the parser refused, a rule reference could not end its
block (rule b { a } was rejected while the same rule over three lines parsed), and the normative grammar
comment documented NOT_IN and a clause-level KEYS, neither of which was ever accepted.

The data loader, where every rule's answer starts

A data file of nothing but comments aborted the process. Exit 101,
panicked at guard/src/rules/libyaml/event.rs:63: not implemented, on four shapes: one comment line, the same
without a trailing newline, several comments with blank lines between them, and a fully commented-out template.
An empty file and a whitespace-only file already exited 255 saying the file is empty, so the degenerate case was
handled and this one was not. The event arriving at the wildcard arm is YAML_NO_EVENT, and it arrives because
the loop had no exit for a stream that never started a document: DocumentEnd was the only arm that returned
and StreamEnd shared the no-op arm, so the next pull ran past the end of the stream and libyaml answered it
with a success. StreamEnd now ends the loop and reports the file as holding no document, worded as the message
an empty file already gets. The wildcard arm remains and no longer aborts; it returns an error carrying the
event type, so a future miscount stays diagnosable.

True and TRUE were read as strings, so a gate never fired. Under
when Properties.PublicAccess == true, against a bucket with Encrypted: false: true, yes and on exit 19
and catch the violation, and True and TRUE exit 0 leaving it unchecked. is_bool_true and is_bool_false
matched lowercase only, while the null arm two lines below lowercased first, so within one function null was
case-insensitive and bool was not. YAML 1.1 makes all 22 spellings boolean; the 1.2 core schema admits six. The
loader already read yes, on and the single letters, which only 1.1 makes boolean, so the vocabulary was
already 1.1's with the capitalised spellings missing, and those four are boolean under both schemas so they were
missing whichever version was meant. test_handle_bool_happy_path has enumerated all 22 as the intended set
since it was written — and could not fail, because its assertion sat inside an if let MarkedValue::Bool(..) and
14 of the 22 cases passed while asserting nothing. handle_type_ref used str::parse::<bool>, so an explicit
!!bool yes was stricter than a bare yes; both now agree. Case-insensitive matching was rejected because
tRuE is boolean under neither schema, and narrowing to the 1.2 set was rejected because it would take yes
and on from documents that write them, which is a separate argument.

A query for Tags was answered out of Tag. A rule asking Properties.Tags.Name of a bucket with
Properties.Tag.Name reported a ComparisonError over /Properties/Tag/Name carrying that value: the rule
named one property and was answered about another. One-way, since Tag against Tags reported
MissingProperty. The cause is the class-case entry in the converter list the lookup falls back to when a key is
absent as written. cruet's class case is Rails' class-name rule, PascalCase and singular, so
to_class_case("Tags") is Tag; every other entry respells a key rather than naming a different one, and
to_pascal_case already covers PascalCase. It is not a plural rule either — Analysis becomes Analysi and
Metadata becomes Metadatum. Removing it costs no reach, measured by instrumenting the fallback: class case
won 57 lookups across the registry's 193 paired files and pascal, title and train produced the identical string
for all 57.

A duplicated key took the last value and said nothing. Encrypted: false followed by Encrypted: true
evaluates as true, the reverse order as false, and {"Encrypted":false,"Encrypted":true} the same way, all
with nothing on stderr. A reviewer reading a template top-down sees the first value and the tool judges the
last. YAML 1.2.2 makes this a loading failure rather than a merge — a mapping's keys must be unique per §3.2.1.1
and §3.2.1.3, and §3.3 lists non-unique keys among the loading failure points — so rejecting the document would
be defensible, and it would also fail every template carrying one today, which is a maintainer's call. This
reports the fact and leaves the verdict: one line naming the key's path and both locations, the last value still
winning, no exit code moved. The duplicate is not visible in the loader, which keys its maps on name and
location so both entries survive; they collapse a layer up where the map is rebuilt on the name alone, which is
also the only point that holds the path the key was reached by. Detection is per-mapping by construction, so a
property name repeated across two resources — which nearly every real template does — cannot register. Counted
before building: one occurrence in 71 corpus files, in the fixture that exists for it, and none in 256 registry
files. cfn-guard test still cannot report duplicates in its embedded inputs, because serde_yaml collapses a
repeated key during deserialization before any of this runs.

An unreachable!() in the FFI error table. get_code in guard-ffi answered Error::InternalError with
unreachable!(), in the table every error crossing the boundary is fed through and the same one that gained a
code for the no-document error above. It is not reachable through cfn_guard_run_checks as the code stands —
that path parses through serde and never touches the libyaml loader, the one construction site that is on the
path is mapped to ParseError first, and all four non-string key shapes come back as ParseError without
panicking. It is fixed regardless: Error and run_checks are public, the remaining variants are constructed
in a module this path avoids only because it writes through a BufWriter of its own, and a table total over
every variant should not rest on an unreachability the next commit can invalidate. The arm gets a code and the
table stays exhaustive rather than gaining a catch-all, since exhaustiveness is what surfaced it.

A leading space before a numeric index or a quoted key was a parse error, while every other bracket form
tolerated one. Names[0] and Names[0 ] parsed; Names[ 0], Names[ 0 ], Names[ -1 ] and
Resources[ "MyBucket" ] did not. This is the [x]-versus-[ x ] defect a second and third time: that fix
added whitespace skipping to the named branch of all_indices and left the numeric index and the quoted key
alone. The tests compare the resulting query part rather than merely that the file parses, because falling
through to a different part is how the first version of this defect did its damage.

Filters and captures in the query engine

[*] and .* both expand a map and hand each entry's value onward. That is right for a collection and
wrong for a single object whose fields the filter names: for
{ Statement: { Effect: Allow, Action: "s3:*" } } the values reaching the filter were the strings Allow
and s3:*, Effect == 'Allow' matched neither, and the filter selected nothing. An assertion over an empty
selection is a SKIP, so the two spellings disagreed — .* failed and [*] exited 0.

A filter capture leaked across iterations of a block. An iteration that captured nothing under a name fell
through to the parent scope, which by then held the earlier iterations' merged keys, so a non-compliant
resource passed on a compliant neighbour's key. Adding a compliant resource is what made the non-compliant
one pass, which is why such a rule looks correct when tested one resource at a time. BlockScope now carries
the capture names its own clauses declare, read out of the rule text rather than learned at runtime — the
first attempt learned them as keys were captured, which made the verdict depend on document order.

A bare bracket is a capture too, and was not counted as one. Properties.Tags[ tk ] written without a pipe
goes through all_indices and becomes AllIndices(Some("tk")), which captures the entry's key at retrieval
exactly as a filter does, but the name collector read Filter and MapKeyFilter only. No block declared the
name, so the lookup deferred to the parent, which held the keys earlier iterations had merged up. A resource
that captured nothing was credited with a previous resource's key and document order decided the verdict: with
the capturing clause in one when block and the reading clause in a sibling when block that only a resource
without Tags enters, one order exits 0 and the other 255. All orders now agree at 19. Pre-existing;
origin/main gives the same three codes, because no scope there held the name at all. The function's own doc
comment claimed AllValues and AllIndices were collected through arms of their own, and there were none.

Resources[*][ nm | ... ] then %nm ended the run at 255 while Resources[ nm | ... ] — the same filter
without the wildcard — worked. After a wildcard the map was already expanded when the filter ran, so the
name was discarded and the capture never made. The filter now runs at the expansion site, where the key is
still in hand.

Reporter aborts and findings with nowhere to go

Four reachable unreachable!() in the CloudFormation reporter and a reachable todo!() in the Terraform one
took the process down at exit 101, the latter on an everyday rule (IN against a plan). One was reachable
only after fixing the panic in front of it, which is why "not reproduced" is not the same as unreachable.

One rule that could not be evaluated discarded the whole file's report, in all four output paths and in
cfn-guard test: a file whose second rule read a variable that does not exist printed one error line and
threw away five real findings from its third rule.

Findings the per-resource output cannot place are reported under their own heading, Findings that belong to no resource:, each entry naming the rule it came from. The existing Could not be evaluated: is for a block
or rule that failed on a condition nobody could decide; these clauses were decided and merely have nowhere to
be printed, and filing a missing property under "could not be evaluated" would report an ordinary failure as
an undecidable one.

Which findings those are is decided by what the reporter rendered, not by predicting it from paths — and
the first version of this section predicted, wrongly, in both directions. It asked once per rule on the
premise that one placed sibling renders the whole rule; pprint_clauses in fact gates every clause
individually, so a rule with one located finding and one pathless one showed the first and lost the second
from the console entirely while the JSON carried its reason. And it could not see a block whose query failed
at the document root, because it required unresolved to be absent and MissingBlockValue is the one of
four block-report constructors that sets it — so a rule querying a top-level property against a
CloudFormation template exited 19 saying nothing, taking the author's own << >> message with it.

rendered_contexts is the union of the resource clause sets, which is by construction what the per-resource
output will show, and every clause and block is asked against it. Deciding per clause without that set
reintroduces a different fault, because the evaluator emits two reports for one comparison it resolved one
way and could not resolve another — the context set is what separates the unresolved twin from a genuinely
unreported sibling. Alongside: the author's << >> message now reaches the section (.or_else over the two
message slots was dead in its second half), NoValueForEmptyCheck no longer prints the negation of what
happened, and a rule name matched by prefix no longer renders one rule's block under another's findings.

Two further gaps in that section, both found by reviewing it rather than by a test. It keyed "was this shown
under a resource?" on the trimmed context string, so two clauses of one rule whose text is identical were
indistinguishable and the placed one made the pathless one count as shown; a rule asserting Properties.Tags exists both inside a resource block and at the document root reported one of the two and printed no section at
all. What separates that from the legitimate twin the context set exists to suppress took establishing: both are
two ClauseReport nodes, adjacent siblings, with byte-identical context and no recorded source span, and what
differs is the check carried, UnResolved beside InResolved for the twin against UnResolved beside
UnResolved for the collision. The gate is now node identity plus that one case named for what it is, and it
suppresses a strict subset of what the old one did, so a finding the old code printed cannot be lost.

And the 320-character cap on an explanation was applied before trimming, so a message's own indentation spent
its budget. A message is written inside the clause carrying it, so every line arrives indented to that clause's
depth, which made the length an author could write depend on how deeply the rule was nested; 9 of the corpus's
233 messages exceed the cap as written and only 5 do once trimmed. It now measures the content, and an
explanation that fits once trimmed comes out whole and unmarked, because an ellipsis is a claim that content was
dropped. One consequence is visible in four registry outputs: a short message no longer prints the whitespace
around it, so the evaluator's sentence joins it a line earlier rather than after a blank one. No exit code moves
and nothing is lost, but the separation between an author's message and the machine's reason was incidental and
is now absent.

Both console reporters also aggregated findings into a std::collections::HashMap and iterated it to write
the report. That hasher is seeded per process, so ten runs of one binary against a three-resource template
produced five distinct outputs.

Parser boundaries

Items[4294967296] wrapped to Items[0] and compared the wrong element. Non-finite float literals (1e400)
and literals that underflow to zero were accepted. Duplicate rule names and repeated parameter names were
accepted silently. A map key that reads as an integer was treated as an index rather than a key.

cfn-guard test reported success for suites it never ran

Three defects in the same family as the rest of this branch, found by following up why an expectation in the
AWS rule registry had never been checked.

A test suite could be run against the wrong rules file, and its failure reported as success. Test files
under a tests/ directory are paired with rules files by prefix, taking the first match in filename-sort
order, so a shorter stem claims a longer stem's tests. With s3.guard and s3_encryption.guard in one
directory, tests/s3_encryption_tests.yml went to s3.guard, which does not define the rule its expectations
name, and s3_encryption.guard was reported as having no tests associated. A suite that expects PASS where the
template violates the rule exits 7 under test -r/-t and exited 0 under test -d over the same two
files. Paired on the longest matching prefix now. The looseness is deliberate and is kept: requiring a _tests
suffix would stop <prefix>.yml and <prefix>-1.json pairing, and a character-class boundary cannot separate
the readings because _ is both the conventional separator and a word character. min_by_key over the
reversed length rather than max_by_key, so a tie still goes to the first file in sort order as it did before;
ties are reachable, since x.guard and x.ruleset share a prefix.

A test file matching no rules file was discarded without a word. That is what a rename leaves behind.
s3_bucket.guard beside tests/s3_tests.yml, left over from when the rules file was s3.guard: test -d
printed only that the rules file had no tests associated, which reads as benign, and exited 0. Renaming the
test file and changing nothing else gives exit 7. Each unmatched file is now named on stderr with its path.

An unchecked expectation was invisible to every structured reporter. An expectation naming a rule the file
does not define is not checked; the plaintext reporter says so on stderr and json, yaml and junit said
nothing. A test file whose every expectation names a nonexistent rule produced
<testsuites tests="0" failures="0" errors="0"> and exited 0, so a pipeline gating on that report had nothing
to gate on. Reported now as a skip in a field of its own, unchecked_expectations, not folded into
skipped_rules -- those are opposite directions of one mismatch, rules with no expectation against
expectations with no rule, and a consumer reading one array could not tell which had happened. A skip and not a
failure, because escalating it changes the exit code of a shipped CLI for every user with a typo in a test
file, and that is a maintainer's call; making the fact visible is not. The field is omitted when empty, so
every report over a suite with nothing unchecked is byte for byte what it was.

Run against the AWS rule registry at 7f7340c, unmodified, the two diagnostics name three test files that
have never run and eleven expectation names that answer to no rule. Two of those three files are the only test
file their rules file has, so s3_bucket_default_lock_enabled and cloudfront_accesslogs_enabled have no test
coverage at all: one filename drops a trailing d, the other begins with a space. Both suites pass once
wired up. Those are fixed in that repository rather than here.

Merging alongside #719

Both PRs merge into origin/main cleanly on their own. Only the combination conflicts, and no single
resolution direction is correct for it — of the five conflicted paths, three want this branch's side, one
wants #719's, and two want a union. One of them is dangerous to get wrong and the damage is silent.

Measured by trial merge into origin/main at 3e265bb, with rerere disabled so a recorded resolution
could not report a false clean, and judged by the merge output text rather than by the unmerged-path list.

path	this branch's side	#719's side	resolve to
guard/src/rules/parser.rs	the 87-line extract_message grammar	the original unbounded find_substring(">>")	this branch
guard/src/rules/path_value.rs	288 lines of QueryResolver and its two select functions	deleted	#719
guard/src/rules/mod.rs	the add_merged_capture_key trait method	deleted	this branch, plus #719's own deletion
guard/src/commands/reporters/validate/common.rs	an import list including FileReport and Messages	the same list without them	this branch (its superset)
guard/src/commands/reporters/test/structured.rs	has_unchecked_expectations and number_of_unchecked_expectations	build_junit_test_cases with an elided lifetime	both

parser.rs is the one that matters. #719 branched before any of the three attempts at extract_message
and its Apply rustfmt commit reformatted the signature, which is what makes it conflict rather than merge.
Its side is the original unbounded search. Resolving that hunk in #719's favour reinstates the defect where one
forgotten >> consumes every following rule as message text and the run reports PASS at exit 0 — the headline
finding of this PR, and silent when it recurs.

path_value.rs is the mirror image: this branch carries that engine only because it does not delete it, and
deleting it is #719's entire purpose. It does not touch the duplicate-key detection added here, which sits
earlier in the file, well outside the conflict region.

The order matters and only one order is dangerous. If #719 merges first, this branch rebases onto it and
the resolution happens here, so no trap reaches anyone. If this branch merges first, the conflict lands in
#719 and the parser.rs hunk is the one that must not be resolved the obvious way.

This table is re-derived rather than carried forward: it named two conflicted paths when this branch was 27
commits shorter, and both the count and the resolution directions have changed since. Re-derive it again if
either branch moves.

A rule could not be composed out of other rules

rule MAIN { H_A H_B } failed whenever a rule it referenced was not applicable. Two helper rules each
guarded by a when on their own resource type, and a template carrying one of those types but not the other,
gave H_A PASS, H_B SKIP and MAIN FAIL at exit 19 — reporting a violation on a template that violates
nothing. A template carrying neither type failed as well.

eval_guard_named_clause claimed every assertion SKIP as FAIL, in both polarities, which also made the arm
below it unreachable for assertions. eval_parameterized_rule_call carried the identical line for r(...),
with a comment claiming to mirror the first; both are fixed together and one test asserts both, so a change to
one cannot pass on the other's coverage.

This is why decomposing a ruleset over disjoint resource types has not been done: neither workaround holds. A
conjunction of references fails whenever any type is absent. A disjunction returns PASS when any one helper
passes, so a template with a violating IAM Role beside a clean DynamoDB table passes a rule that should fail
it — a false negative in a compliance rule. The AWS rule registry's existing helper idiom composes with OR
only because its helpers are alternative paths to one conclusion, and that does not transfer.

A negated reference is included: rule deny when Resources.*.Type exists { not inner } failed on a template
with one S3 bucket and no KMS key. It now reports the rule did not apply because it referenced rule [inner], which did not apply to this input. A negated reference used as a when gate is deliberately unchanged,
because a gate that closes silently disables the rule it guards; the registry's one negated reference is a
gate, and that decision is what protects it.

Exit codes said the tool broke when the rules file was wrong

Two paths reported an authoring mistake as exit 255, the internal-failure code, where 5 means rules-file
error:

• parse-tree on a rules file it cannot parse — validate said 5 for the same file, so two subcommands
  disagreed about whose fault it was.
• validate on a rules file referencing a variable no let declares. Forgetting a let is a common mistake,
  and the caller was told the tool had failed. The undeclared name is now reported with Could not resolve variable by name <name> across scopes, so it can be acted on.

The reason a 255 survived here is a test that passed for the wrong reason. guard/tests/parse_tree.rs had two
cases sharing one expected output that is an I/O error. One points at dne.guard, absent on purpose. The
other pointed at validate/rules-dir/malformed-rule.guard while the file lives at
validate/malformed-rule.guard — one directory up. So it failed at File::open, matched the shared I/O
string, and passed green. parse-tree's parse-error exit code therefore had no coverage at all. The fixture,
never having been run, also turned out not to contain a syntax error: it is an undeclared-variable file, which
is how the second defect above came to light. The repository had no genuinely malformed rules fixture; there
is one now, the cases assert what their names claim, and the exit codes are documented, which they were not.

A test expectation could name a rule that does not exist

cfn-guard test printed No rule named X is in this file, so its expectation was not checked and exited
0. An expectation naming a rule that is not there asserts nothing, so a rule renamed without its test file
being updated leaves a test that silently stops testing while the suite stays green.

It is now exit 1 — TEST_ERROR_STATUS_CODE, not TEST_FAILURE_STATUS_CODE, following the line this
repository already draws at guard/tests/test_command.rs: "an expectation that could not be evaluated is a
different answer from an expectation that was not met." All four output formats report it; junit uses its
<error> element, which consumers treat as red exactly as they treat failures.

The question that decided whether a blanket failure is right is whether an expectation for a rule in a
sibling rules file can legitimately be checked in directory mode. It cannot: OrderedTestDirectory::from
reduces the candidate rules files with min_by_key, which yields exactly one claimant, so a test file is
never evaluated against a second rules file. A directory whose test files are named to cover several rules
files at once will now go red, and it was never checking what it appeared to.

This has a consequence outside this repository, measured rather than assumed.
aws-cloudformation/aws-guard-rules-registry has 30 such expectations in
rules/aws/aws_cloudformation/tests/cfn_no_explicit_resource_names_tests.yml, across 11 distinct names, in a
suite that reports clean. Its .github/workflows/{ci,build,publish}.yml each install cfn-guard unpinned
from main and run cfn-guard test -d ./rules/, so when this change is released that repository's CI and
its publish pipeline go red until those expectations are dealt with.
aws-guard-rules-registry#286
removes them and is green under every cfn-guard version, released or not, so landing it first leaves no red
window in either repository.

A regular expression's own diagnostic was built and thrown away

Both failure returns in parse_regex_inner were nom::Err::Error, which alt treats as recoverable, so the
fancy_regex message was constructed and discarded and the next value production's complaint was reported
instead. == /a(/ exited 5 saying expecting either a property access "engine.core" or value like "string",
with no mention of the parenthesis. It now says Could not parse regular expression: Parsing error at position 2: Opening parenthesis without closing parenthesis. The file's own convention two functions above already
required Failure here and cross-references parse_range; two sites followed it and these did not. Two
existing tests pinned the Error variant by assert_eq!, and moving those assertions is the point of the
change rather than collateral damage.

Two changes that alter what a rules file means

Both are deliberate, both are measured, and neither is a bug fix you can take without noticing it.

A string or regex literal ending in a backslash. parse_string_inner and parse_regex_inner each decided
whether a delimiter was escaped by testing whether the text in front of it ended with \, which cannot tell
\ from \\. When it guessed wrong the scanner resumed past the closing delimiter, so the next quote or
slash anywhere in the file became the terminator — an apostrophe in a comment was enough — and every clause
and rule in between was absorbed into a string value, unevaluated and unreported, at exit 0.

Fixing it means a backslash consumes the character after it, whichever character that is. That is the rule
every language with a backslash escape uses, and it makes a backslash's meaning independent of its position.
The consequences: in a string "a\\b" is now one backslash rather than two, and in a regex \\ closes
itself so a following / ends the literal. There is no reading in which \\/ both terminates in /^x\\/
and does not terminate in [A-Za-z0-9\\/+=] — the swallow and the fixture are one construct, so one of them
has to move.

Measured blast radius: \\ inside a string literal, a backslash immediately before a quote, and \\/
inside a regex each occur zero times across all 313 .guard files and 435 KB in this repository and the
AWS rule registry. The two exceptions are the two copies of advanced_regex_negative_lookbehind_rule.guard
here, whose \\/ is rewritten to \/ — the spelling that always meant what the file intended. A rules
file in the wild using \\/ in a regex now fails loudly at parse rather than misreading silently.

Mixed range bounds, r[0,20.5], are now accepted — and that commit is separable. It is not a defect fix
and can be dropped without touching anything else. docs/CLAUSES.md promises that integer and float
"compare against each other as numbers. That includes range membership", but its examples are r[5.0, 100.0]
and r[5, 100] — homogeneous bounds with a mixed value. Mixed bounds are not promised, and RangeType
holds one type, so refusing them was a representable-shape limitation rather than a defect. The case for
accepting them is that Size in r[1,2] holds for a Size of 1.5 while Size in r[0,20.5] did not parse at
all, so the gate was narrower than the thing it gated; the integer bound widens to float and is refused if
widening would move it. Accepting more input invalidates no existing file. But it changes the accepted grammar
of a public DSL, which is a maintainer's call, so it is its own commit.

Verification

cargo test --release --no-fail-fast: 1489 pass, 0 fail, 2 ignored across 16 targets, from 1379 at the
branch point. cargo clippy --release --all-targets -- -D warnings and cargo fmt --all -- --check both
clean.

The whole blast radius across both repositories, and every item in it is intended:

• Parse trees. Every .guard file parsed with the branch point's binary and this one, compared on exit
  code and on the sha of the full tree. 210/210 in the AWS rule registry identical. 105 of 108 here identical;
  the three that differ are the deliberately-unparsable fixtures going 255 to 5, with byte-identical tree
  output — the exit-code fix, not a parse change.
• The registry's own suite. All 44 directories with a tests/ subdirectory, run as
  cfn-guard test --dir at clean upstream, before and after. One changed: rules/aws/aws_cloudformation,
  0 to 1, stdout unchanged. That is the ghost-expectation fix firing on the 30 dead expectations described
  above, and nothing else in the registry moves.
• Snapshots, not the working tree. Both sweeps read git archive snapshots at explicit refs rather than a
  live checkout, after two measurements in this work turned out to be reading a tree that had moved
  underneath them.
• A 440-cell oracle matrix over operator x operand-shape x polarity x position, whose expectations come from
  the language semantics rather than from the binary.
• A 1,626-pair sweep for the rule-reference change specifically — 193 registry cfn-guard test pairs plus
  1,433 validate pairs — byte-identical, with sensitivity proved rather than assumed by showing a fixture in
  the same shape does move across the two binaries.

A caveat on all of it, worth more than the numbers. These are regression instruments. Every one of them
reported zero changes for most of the parser fixes here — correctly, because no fixture contains a forgotten
>>, or a string ending in a backslash, or a transposed range. A differential answers "did any of these
inputs change answer", not "is there an input that gets a wrong answer". Every defect in this branch was
found by reading what a function commits to and then constructing the input that violates it, and each fix
carries a test built the same way.

Two flavours of empty selection answered differently

rule H { Resources.*[ Type == 'AWS::DynamoDB::Table' ].Properties.TableName not exists }

document	before	after
{} — no Resources key at all	PASS	SKIP
{"Resources":{}}	PASS	SKIP
one S3 bucket, no DynamoDB table	SKIP	SKIP
one clean DynamoDB table	PASS	PASS
one violating DynamoDB table	FAIL	FAIL

Rows 1 and 2 are the same logical situation as row 3 — an empty selection with not exists over it — and
answered PASS where row 3 answered SKIP. A rule about DynamoDB tables reported compliance for a document
containing nothing, and correctly reported itself inapplicable for a document containing one unrelated resource.
Across sixteen operator spellings twelve disagreed, and the disagreement was not uniformly permissive: for
exists, !empty, == and in the empty-document flavour answered FAIL where the other answered SKIP. The
highest-stakes row was a when condition, where a not exists gate opened on one flavour and closed on the
other — changing whether the guarded body ran at all rather than only what was reported.

Two fixes were implemented and rejected before this one, both caught by measurement. Treating any
all-unresolved result as an empty selection turns Resources.*.Properties.BucketEncryption exists from FAIL into
SKIP — "this bucket has no encryption configured" is the entire point of a compliance rule, so that reading gives
up the tool's main job. Keying off "an expansion with query parts after it" produced every target row correctly
and then failed four existing tests that encode requirements rather than the defect: an IAM role carrying no
Tags must fail Properties.Tags[*].Value == /.../, and some Tags[*].Key == /PROD/ over Tags: [] must fail.
Those four tests are unchanged.

The predicate that satisfies all of it: an empty expansion yields an empty selection only when a filter is
still pending at or after the point the collection came up empty. A filter is how the language says "the subset
of things this rule is about", so a filter that has not yet run over an empty input means the rule has no
subjects. A bare wildcard is a projection over a property of a subject already chosen, so an empty projection is
an answer about that subject. Searching from the current index rather than from zero is what keeps
Resources.*[ Type == 'X' ].Properties.Tags[*].Key == /PROD/ failing on an empty tag list — that filter has
already run, the resource was selected, and its empty Tags belongs to it.

This routes the empty-collection case into the SKIP path that already exists for filters, which
eval.rs's own comment describes as happening "when the query has filters in them". QueryPart::Index is
excluded deliberately: Items[0] names one element rather than enumerating a set.

Blast radius: all 44 directories of the AWS rule registry run as cfn-guard test --dir, and every one of its
210 rules files run against three empty-ish documents — 630 verdict pairs, zero changes. Every registry rule
is gated by when %var !empty, so none takes the path this changes; the defect was reachable by hand-written
ungated rules.

Known limitations

Reviewing this branch found more than reading it did

Later passes over this branch's own commits turned up four things that no amount of reading the diff would
have reached, because each only exists when a documented command is actually run. They are worth listing
separately because three of them are in the tooling around the change rather than in the change.

cargo test --all aborted on this branch. Fifteen of the 34 tests that parse at or near
MAX_NESTING_DEPTH exceeded libtest's 2 MB thread stack in the default debug profile — SIGABRT, cargo exit
101 — while --release passed. pr.yml runs cargo test --verbose, which is debug, on ubuntu, macos and
windows, so this was red on three platforms. The deep parses now run on an explicitly sized thread, so they
no longer depend on the harness's default. What hid it is worth naming: MAX_NESTING_DEPTH's doc comment
justified the bound with "about 2.3x to spare", measured on an optimized build and written without saying so.
Unoptimized, the same ladder tops out below the bound. Every stack figure in that comment now states its
profile, and the argument for 128 rests on the corpus instead, because a corpus fact survives a change of
profile.

cfn-guard test exited 0 on a rules file it could not parse, in json, yaml and junit, while
printing errors="1" in the same run. A CI job gating on the exit code saw success on a broken ruleset,
which is the worst failure direction available for a policy tool. single-line-summary already exited 1 on
the same bytes, so the two format families disagreed. All four now route through the same
TestResult::get_exit_code() rather than assigning codes by hand at each site.

The four output formats disagreed about content and about codes in three more places, found by
enumerating the grid rather than by fixing the reported case: a stale-rule-name diagnostic was dropped in
single-line-summary when the same case also carried an unparseable expectation string, and the
directory path answered 7 where three formats and the single-file path answered 1 for identical input. There
are now tests asserting that every format agrees over one rules file and over a directory — the invariant,
not an instance. Both of the earlier fixes here had been to an instance, and one of them created the same
defect pointing the other way.

The Windows registry CI step decided pass/fail from stdout. if (<command>) in PowerShell tests the
command's output for truthiness, not its exit code, and that command prints thousands of lines — so the step
logged "have passed" for runs that had exited non-zero, and would have failed a successful run that printed
nothing. Its bash twin uses if <command>; then, which does test exit status, so the two legs of one check
were checking different things. Both now test $LASTEXITCODE. The same expression appeared in the
parse-tree loop, where it was correct only by luck.

Four doc comments also stated numbers that could not be reproduced, which matters here because a false
comment in this area misdirected several of these investigations. The corpus depth maximum was 6 and is 7,
reached by two files rather than four; the loader's two corpus figures were each high by one, and one of them
named "the deepest CloudFormation template" in a corpus that contains no template files. A ratio was still
derived from a threshold that had already been corrected. And a call-graph component count was removed rather
than corrected: the figure depends on how a parser passed to a higher-order combinator is attributed, four
defensible models give four different answers, and the sentence named none — so no reader could check it. Two
user-facing refusal messages that quoted corpus statistics now make their point qualitatively, because a
message asserting a maximum over a repository whose release cadence this one does not control is wrong again
on someone else's schedule.

Three questions are left open on purpose, each measured and recorded rather than settled by accident: a key
filter after [*] versus .*; what a filter applied to an already-indexed value (Rules[0][ ... ]) should
mean; and ::MODULE stripping in a type-block name. Each needs a decision about what the language means, and
guessing one is worse than leaving it visible.

[awsmadi]

@Zee2413 @satyakigh — worth a look ahead of the other open PRs, for two reasons.

CI is fully green at 05edeef (run 34234046182): all 16 jobs, including the three aws-guard-rules-registry integration jobs and Testing Windows Install Script (install-guard.ps1). That Windows job is currently the only failing job on main (run 34151842199 at 814bd00a).

This PR also adds .github/scripts/check-registry-corpus.sh, which asserts the pinned registry corpus is in its known state instead of demanding an exit code the corpus cannot produce. That is what clears #720. #720's integration jobs fail for a reason that is not a defect in it: it escalates unchecked test expectations from a stderr note to a hard error, and the pinned corpus carries 30 of them in rules/aws/aws_cloudformation/tests/cfn_no_explicit_resource_names_tests.yml. That suite asserts 11 rule names (static_name, static_rolename, no_static_names, and eight others) that its .guard never declares; the file declares only CFN_NO_EXPLICIT_RESOURCE_NAMES. Verified by building #720's binary, running it against the pinned corpus (exit 1), removing exactly those 30 expectation lines, and re-running (exit 0). So #720 needs no code change, only this harness.

One other thing only a maintainer can unblock: the registry-side fix for those stale expectations is aws-cloudformation/aws-guard-rules-registry#288, whose CI has never run because it is sitting in action_required awaiting workflow approval.

[satyakigh]

Two actionable issues at current head 9a4cc06:

• Blocking — valid JSON integers above i64::MAX still sign-flip on the public run_checks path. guard/src/rules/values.rs fixes the serde_yaml::Value arm by preserving the digits, but the serde_json::Value arm still does num.as_u64().unwrap() as i64. run_checks parses valid JSON through that arm before its YAML fallback. I reproduced 18446744073709551615 with a rule Size < 0: the rule reports PASS; changing this arm to preserve the decimal string makes it FAIL. Please apply the same fix to the JSON arm and add a regression through run_checks (which also covers Lambda/FFI callers).
• FFI error-code contract is incomplete. get_code now returns 25 for UnsupportedDocument and 26 for UndecidableComparison, while guard-ffi/src/lib.rs still documents 1..=24, and every_error_code_is_distinct_and_usable omits both constructible variants. Please update the documented range and include 25/26 in the distinctness test.

[awsmadi]

Follow-up, since the branch and its CI both moved after my comment above.

The branch was force-pushed to rewrite one commit message. The rewrite was message-only: trees are byte-identical between the old and new tips, and git diff between them is empty. So no check outcome can differ because of it. Current head is 9a4cc06d.

Current CI, run 34248709375: 17 pass, 1 fail. The single failure is Testing Windows Install Script (install-guard.ps1), and it is not caused by anything in this PR:

• The same job fails on main — run 34151842199 at 814bd00a — with the same error.
• install-guard.ps1 queries api.github.com/repos/aws-cloudformation/cloudformation-guard/releases/latest unauthenticated, so from a shared runner IP it hits the API rate limit. The job dies in about 20 seconds.
• That script installs the last published release. It never reads this PR's source and never compiles the crate.

Everything else is green: all three aws-guard-rules-registry integration jobs, all three unit-test platforms, both fuzz targets, clippy, fmt, shellcheck, typos, and both install-guard.sh jobs.

My earlier 16/16 figure was measured on run 34234046182 at 05edeef8, which still reports success — that commit is simply no longer at the branch head.

Neither I nor the author can re-run the failing job; we do not have admin on this repository. So it stays red until someone who does re-runs it, or a later run gets past the rate limit.

[awsmadi]

Both addressed at 8447892. Thanks for reproducing the integer case — that made it quick to confirm.

The JSON arm. Reproduced your case before changing anything: { "Size": 18446744073709551615 } against rule r { Size < 0 } through run_checks returned "status": "PASS" with r listed as compliant. You were right about the path, and about why the earlier fix missed it — run_checks tries serde_json::from_str first and only falls back to serde_yaml when that fails (commands/helper.rs:51), and that integer is valid JSON, so it never reaches the arm that was fixed.

The JSON arm now keeps the decimal string, matching convert_yaml so the two agree on the same input. The regression goes through run_checks as you asked, in guard/tests/functional.rs, which is the surface cfn_guard_run_checks and the Lambda handler share. It fails with PASS before the change and passes after.

One thing I checked while there and deliberately did not change: the float arm beside it has no finiteness gate, unlike the YAML one. serde_json refuses a non-finite float while it is still text — 1e400 is number out of range at parse time, measured — and JSON has no .inf or .nan spelling, so a serde_json::Number reaching that arm is always finite and the gate would be dead code. The YAML arm needs one because the libyaml loader does accept those spellings. There is a comment recording that so the asymmetry does not read as an oversight.

The doc comment on try_from_json used to cite this sign flip as one of three reasons not to adopt serde_json. That reason no longer holds, so it is gone; the other two stand.

The FFI contract. Also correct, and the documented range was worse than stale — it told C callers that 1..=24 was exhaustive while get_code could return two values outside it. The header now says 1..=26 and names both. every_error_code_is_distinct_and_usable covers them: both carry a String, so neither had the unconstructible-payload reason the five absent variants have, and the docstring now says so rather than leaving the omission unexplained. They are also the two closest to a collision, being the highest codes and sitting past the 23/24 input codes.

Measured on 8447892: cargo test --workspace is 3196 passed / 0 failed locally, and both CI workflows are green — all 16 jobs on the Pull Request workflow, including the Windows install-script job that had been failing.

[awsmadi]

Measurements on numeric scalar handling

What this is, since it is posted on a pull request and is mostly not about it. Parts 1, 2, 4 and 5
describe behavior present at the 3.2.1 tag and at main, independent of this branch. Part 3 reports
what a candidate fix for part 2 costs, measured, and the numbers argue against making it. Where this pull
request is relevant it is said in place: part 1 notes that this branch does not fix the result it
describes, and part 5 notes which of two representations the branch keeps and why.

Two questions are left open rather than answered — one at the end of part 5, one in its own section
before the reproduction steps. Both need knowledge of intent that is not derivable from the code, so
they are for the maintainers rather than for this review.

Posted here because the measurements came out of this branch and the reproduction steps are easiest to
follow beside it, not because any of it is a claim about the diff under review.

All figures below were produced by building two references from source and running the binaries:

• the 3.2.1 tag, which is the current release
• this repository's main at 814bd00a4e6d761e8b5c9f615dd510b1a2a7c374

Both are named exactly because a fork's main can sit behind this one, and an earlier draft of these
figures was taken against such a fork before the divergence was noticed. Every [upstream] marker
below means verified present at the 3.2.1 tag, at that main commit, or both. [unmerged] means the
code exists only in an open pull request and is named where that distinction changes the reading.

Every command was run twice: once with a value that must reach the comparison, as a control that the
rule and the assertion can produce a failure at all, and once with the value under test.

---

The three questions, up front

These are the parts nobody outside the project can answer. Each is discussed where it arises; they are
collected here so they are not buried.

• Does suppression depend on an incompatible pair answering "no"? If it does, the 131 moved
  suppression expectations in part 3 are a correctness problem becoming visible rather than one being
  caused. If it does not, that change is wrong as written. This is the most useful thing in the report
  and the one part left open. Detail in The question this cannot answer, below.
• Which of the two harms should the tool prefer, for an integer above i64::MAX: a comparison that
  refuses to answer, or one that answers from a value the document does not contain? Detail in part 5.
• Is making the value space hold these integers exactly worth doing, or is the range far enough
  outside real templates that either reading is acceptable? Detail in part 5.

---

1. A JSON integer above i64::MAX reaches the comparison as a negative number

guard/src/rules/values.rs [upstream], in impl TryFrom<&serde_json::Value> for Value:

} else if num.is_u64() {
    //
    // Yes we are losing precision here. TODO fix this
    //
    Ok(Value::Int(num.as_u64().unwrap() as i64))

as i64 reinterprets the bit pattern rather than converting, so 18446744073709551615 arrives as
-1. The comment reads it as a precision loss; it is a sign change.

This arm is reached from validate_and_return_json, exported as cfn_guard::run_checks
(guard/src/lib.rs) [upstream], which tries serde_json::from_str first and falls back to
serde_yaml only when that fails (guard/src/commands/helper.rs) [upstream]. An integer above
i64::MAX is valid JSON, so it takes the JSON branch. cfn_guard_run_checks in guard-ffi and the
handler in guard-lambda both call that function. The cfn-guard CLI does not; it reads documents
through a different loader and is unaffected.

Reproduction

Rule:

rule r when Mode > 1000 {
    Public == false
}

Called through run_checks with verbose = false:

data	3.2.1 [upstream]	main [upstream]
{ "Mode": 5000, "Public": true } — control	"status": "FAIL"	"status": "FAIL"
{ "Mode": 18446744073709551615, "Public": true }	"status": "SKIP"	"status": "SKIP"

The control confirms the rule and the assertion can report a failure.

18446744073709551615 > 1000 is true, so the condition holds, the block is entered and
Public == false is evaluated against true. That is the FAIL the control shows. With the value
under test the condition is evaluated as -1 > 1000, which is false, so the rule does not apply, the
block is not evaluated, and the returned status is SKIP.

The same rule read by the CLI reports FAIL on an equivalent YAML document, so the two entry points
disagree on the same rule and the same value.

Note on the open pull request

An open pull request changes this arm to keep the decimal digits instead of casting. That removes the
sign change, and for a bare assertion such as Size < 0 it changes a wrong PASS into a FAIL. It
does not change the result above: the value becomes a string, an ordering comparison between a
string and a number has no arm in compare_values, and the condition again does not hold, so the
status is still SKIP. Measured on that branch, the SKIP cell is unchanged. This report should not
be closed by that pull request merging.

---

2. KNOWN_ISSUES.md item 2 does not describe the behavior inside a when condition

docs/KNOWN_ISSUES.md item 2 [upstream] reads:

When performing != comparison, if the values are incompatible like comparing a string to int,
an error is thrown internally but currently suppressed and converted to false to satisfy the
requirements of Rust's PartialEq. We are tracking to release a fix for this issue soon.

That describes what the comparison returns. What it does not describe is that the consequence differs
by where the clause sits, and the two differ in whether the run says anything at all.

As a bare assertion, an incompatible pair fails the clause and the report names it, so an author sees
the mismatch and can correct the rule.

Inside a when condition, the same pair means the condition does not hold. eval_rule maps every
non-PASS condition to SKIP for the rule [upstream], so the rule does not apply, the guarded
block is not evaluated, and the run exits 0. Nothing is written to either output stream about the
block that did not run. An author reading item 2 would expect a visible failure and would get silence.

This is a documentation observation, not a second defect: the behavior follows from item 2 plus the
condition-to-SKIP mapping. It is raised because the combination is what a rule author meets, and
because item 2 is the page they would read first.

---

3. Measured cost of treating an incompatible pair as undecided (143 registry expectations move)

3. Measured cost of treating an incompatible pair as undecided

One candidate fix for the when-condition behavior in part 2 is to treat a refusal to compare two kinds
as an unevaluatable clause rather than as a decided "no", so a condition that cannot be answered fails
its rule instead of leaving it not applicable. In this repository's own terms that is adding
Error::NotComparable(_) to the match in is_unevaluatable (guard/src/rules/eval.rs) [upstream],
which today matches Error::IncompatibleError(_) alone; ClauseRole::is_strict beside it already
carries the FAIL-rather-than-SKIP decision.

That was implemented and measured. The numbers are given here because they argue against it, and
because they are reproducible with nothing but this repository and the registry.

Against the aws-guard-rules-registry corpus at commit
7f7340c26ae5d5e8874651dbffeb12e0e9f505b6, running cfn-guard test -d rules over the whole corpus:

• 143 rule expectations move. 131 are Expected = SKIP, Evaluated = FAIL; 12 are
  Expected = PASS, Evaluated = FAIL.
• The comparison that produced those figures was a diff of the corpus report between a binary built
  with the change and one built without it. Both runs exit 1 at this corpus commit for an unrelated
  reason — 30 expectations name a rule their own file does not define — so the exit code alone does not
  separate them and the report has to be diffed. A helper that does the diffing arrives with the
  pull request these figures come from; it is not in this repository yet, and nothing here depends on
  it. The 143 is reproducible with diff over two saved reports.
• The corpus commit above is pinned in the pull request's workflow [unmerged], not in this
  repository's main [upstream], where the registry checkout tracks the default branch. Either way
  the corpus lives in a different repository and its expectations cannot be corrected from here.

The 12 PASS → FAIL cells are the ones worth naming, because a rule whose expectation is PASS
describes a template the registry asserts is compliant, and it is now reported as a violation. Named
cases include:

• LAMBDA_NO_WILDCARD_PRINCIPALS, on the case named "Lambda permission policy limited to an account
  as the principal, PASS"
• IAM_USER_LOGIN_PROFILE_PASSWORD_RESET_RULE, on "IAM User with Login profile and PasswordReset set
  to true"
• SECURITY_GROUP_EGRESS_ALL_PROTOCOLS_RULE and SECURITY_GROUP_INGRESS_ALL_PROTOCOLS_RULE
• EMR_SECURITY_CONFIG_ENABLED_AND_CONFIGURED_RULE

The 131 SKIP → FAIL cells are suppression cases, including several IAM_USER_LOGIN_PROFILE_* tests
named "Guard suppressed" and "CFN_NAG and Guard suppressed".

An independent check inside this repository agrees. A test that holds every operator and operand shape
against a stated oracle [unmerged] reports 24 cells moving onto the disagreeing-with-oracle list
and none coming off it. Among them are == 50 against the string "50" and > 10 against the
string "50", in a condition role — a numeric-shaped string against a number, which is a shape rules
use. The oracle is this repository's own statement of the right answer, so those 24 are wrong by the
project's own definition rather than by an outside judgment.

---

4. A numeric-looking mapping key is addressed differently by the two loaders

4. A numeric-looking mapping key is addressed differently by the two loaders

Independent of everything above, and [upstream] at both references.

cfn-guard validate reads a document through the libyaml wrapper; cfn-guard test and run_checks
read one through serde_yaml. The two resolve a numeric-looking mapping key differently, and because a
resolved key is rendered back into the string a rule addresses, the property's name differs by entry
point.

Measured with 0755 as a key:

entry point	key as addressed
cfn-guard validate	755
serde_yaml-backed readers	0755

So a rule referencing that property resolves under one command and finds nothing under the other.

Not every numeric-looking key does this. 0x1F is resolved to 31 by both, so it is addressed
identically either way; the divergence needs a spelling the two disagree about, and a leading-zero
decimal is the one measured here.

The trigger is a mapping key whose text the two loaders resolve differently, and the failure is a rule
that finds nothing rather than one that matches wrongly. Included because it has the same root cause as
the rest of this report — two loaders reading one document — and because it is not visible from either
loader on its own.

---

5. An integer above i64::MAX has two representations and both are wrong

5. An integer above i64::MAX has two representations and both are wrong

MarkedValue::Int is an i64 and no f64 holds u64::MAX, so a value in that range cannot be
carried exactly by anything the value space currently offers. Both available readings were built and
measured.

Resolved to a float, which is what 3.2.1 and main do [upstream]: ordering answers correctly,
and identity does not. 9223372036854775809 and 9223372036854775810 both become
9.223372036854776e18, so a rule asserting the two differ is reported as a violation. Measured on
builds of both, A != B over those two values exits 19, and the report reads
provided value [9.223372036854776e18] did match expected value [[9.223372036854776e18]].

Kept as the source text: identity is exact and the two values above compare as different, which is
correct. An ordering comparison against a number then has no arm in compare_values, so as a bare
assertion the clause fails closed and, as a when condition, the rule does not apply and the guarded
block is not evaluated — the behavior described in part 2.

Neither avoids both. The only reading that does is one the value space can hold exactly, which is a
change to PathAwareValue and everything downstream of it — comparison, the serializers and SARIF —
and therefore not something to decide from outside the project.

An open pull request currently keeps the text, on the reading that a reported violation which does not
exist costs a reader more than a rule that declines to answer. That is a choice between two harms made
by someone who does not own the value space, which is why it is raised here rather than settled there.
It also has a cost worth stating plainly against itself: resolving to a float instead would fix the
run_checks result in part 1, because a float compares against a number numerically. So the reading
that is wrong about identity is the one that answers part 1 correctly.

Two questions:

• Which of the two harms should the tool prefer, if it has to choose?
• Is making the value space hold these integers exactly something worth doing, or is the range far
  enough outside real templates that either reading is acceptable?

---

The question this cannot answer

The 131 suppression cells admit two readings, and they lead to different work:

1. Suppression depends on an incompatible pair answering "no". If so, those 131 are a correctness
   problem becoming visible, and the classification change is surfacing it rather than causing it.
2. Suppression is independent of that, and the change breaks it. If so, the classification change is
   wrong as written and the fix belongs elsewhere.

Distinguishing these needs knowledge of how the suppression mechanism is intended to interact with an
unanswerable clause, which is not derivable from the corpus or from this repository. It is the most
useful thing in this report and the one part left open.

---

Reproducing every measurement above

Reproducing

Every measurement in parts 1 through 5 is reproducible with nothing but this repository and the registry
repository. The one exception is the operator-matrix count inside part 3, which comes from a test that
exists only in the pull request; it is noted again at the end and is offered as corroboration rather
than as a step to follow.

Parts 1, 2, 4 and 5. Build the 3.2.1 tag. For part 1, call cfn_guard::run_checks with the rule
and the two JSON documents given there, once with 5000 as the control and once with the wide integer,
and compare the status field. For part 2, run the same rule shape through cfn-guard validate on an
equivalent YAML document and note that it reports FAIL where run_checks reports SKIP. For part 4,
put 0755 in as a mapping key and read the same property through validate and through test. For
part 5, put 9223372036854775809 and 9223372036854775810 in one document and assert A != B.

Part 3. Check out aws-guard-rules-registry at 7f7340c26ae5d5e8874651dbffeb12e0e9f505b6. Run
cfn-guard test -d rules from the registry root, saving the output, once with a binary built from
main and once with a binary carrying the change described in that part.

Stated in this repository's own names: is_unevaluatable in guard/src/rules/eval.rs [upstream]
currently matches Error::IncompatibleError(_) alone, and ClauseRole::is_strict beside it is what
decides that an unevaluatable clause fails rather than skips. Adding Error::NotComparable(_) to that
match is the change — a refusal to compare two kinds becomes an unevaluatable clause, so a condition
holding one fails its rule instead of leaving it not applicable.

Then diff the two saved reports. The 143 figure is the count of Expected = ..., Evaluated = ... lines
that appear in the second and not the first. Both runs exit 1 at this corpus commit for the unrelated
reason noted above, so the exit code is not the signal.

One precision about that figure: it was measured on the pull request's branch, where the same decision
is reached through an intermediate classification the branch adds, rather than by editing
is_unevaluatable on main directly. The two are the same behavioral change and the count is expected
to match, but only the first was run. If the number differs on main, the main number is the correct
one.

Part 3, the operator matrix. That result comes from a test that exists only in the pull request, so it is
offered as corroboration rather than as something to reproduce here.

[awsmadi]

Thanks for both of these — you reproduced two real defects, and I owe you an apology for the silence. Both were fixed the day after your review and nobody told you, so from your side this has looked like feedback into a void for three days.

JSON sign flip — fixed in 72ff532f. values.rs now gives the serde_json::Value arm the same digit-preserving treatment the YAML arm had. Regression goes through cfn_guard::run_checks as you asked, since that is the path Lambda and FFI callers take: a_json_integer_above_i64_max_keeps_its_digits_exactly in guard/tests/functional.rs. Reverting only that one line flips Size < 0 from FAIL back to PASS, so the test is not vacuous.

I also enumerated the class rather than just fixing the reported member. values.rs has three as_u64 sites: the two is_u64 arms, both now fixed, and number_key_name, which never had the cast. Repo-wide, is_u64() occurs at exactly those two places, and every other as i64 is float-to-int truncation that saturates rather than reinterpreting a bit pattern, each already range- or finiteness-guarded. Different class.

FFI error codes — fixed in 84478921 and d3c9d709. The header documents 1..=26, skipping 19, and both variants are in every_error_code_is_distinct_and_usable. Worth noting why the second commit exists: the distinctness test catches a plain collision but cannot see an identity swap, and the exact-value test catches the swap but not the collision. They are complements, so neither is redundant.

Verifying the above turned up four defects of our own, now fixed on this branch:

• 64b5aefe — 72ff532f updated one copy of a now-false comment and left another in strings_tests.rs still claiming the JSON arm was unfixed.
• 2d474b52 — a guard added to stop the loader-divergence set growing quietly was assert_eq!(3, DIVERGENT.len()) over a [&str; 3], i.e. assert_eq!(3, 3). Adding a case without bumping the declared length is a compile error, so the only edit that reached the assertion was one that grew both — the edit someone doing the bookkeeping deliberately has already made. The case list is gone; there is one list now.
• 1ac75f7e — resolve_int's doc said a decimal with a redundant leading zero stays a string, while the code has no such branch and an inline comment eighteen lines below says otherwise. from_str_radix("0755", 10) is 755.
• c8fe2ca7 — two references my own rename broke.

[awsmadi]

Closing in favor of the smaller single-fix PRs the maintainers asked for. The work on this branch is being split into one PR per fix, submitted a few at a time in severity order; the first batch is #747, #748, #749, and #750. The branch stays on the fork unchanged as the integration reference, so anything here that has not yet been resubmitted is still reachable.